-- Logs begin at Tue 2022-08-09 15:25:20 UTC, end at Tue 2022-08-09 17:40:40 UTC. -- Aug 09 16:30:46.934867 dv-cinder-831247-22-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 9 16:30:46 2022] *** Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: nodename: dv-cinder-831247-22-iscsi Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: machine: x86_64 Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: clock source: unix Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: pcre jit disabled Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: detected number of CPU cores: 4 Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: current working directory: / Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: detected binary path: /usr/bin/uwsgi-core Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: your processes number limit is 47680 Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: your memory page size is 4096 bytes Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: detected max file descriptor number: 2048 Aug 09 16:30:46.945993 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: lock engine: pthread robust mutexes Aug 09 16:30:46.947195 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: thunder lock: enabled Aug 09 16:30:46.947195 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 09 16:30:46.947195 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 09 16:30:46.985196 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: Python main interpreter initialized at 0x55d9d6a379c0 Aug 09 16:30:46.985196 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: python threads support enabled Aug 09 16:30:46.985196 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: your server socket listen backlog is limited to 100 connections Aug 09 16:30:46.985196 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: your mercy for graceful operations on workers is 90 seconds Aug 09 16:30:46.985575 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: mapped 403077 bytes (393 KB) for 2 cores Aug 09 16:30:46.985651 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: *** Operational MODE: preforking *** Aug 09 16:30:46.985890 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: *** uWSGI is running in multiple interpreter mode *** Aug 09 16:30:46.985890 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: spawned uWSGI master process (pid: 86830) Aug 09 16:30:46.986124 dv-cinder-831247-22-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 09 16:30:46.988082 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: spawned uWSGI worker 1 (pid: 86843, cores: 1) Aug 09 16:30:46.988082 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: spawned uWSGI worker 2 (pid: 86844, cores: 1) Aug 09 16:30:46.988082 dv-cinder-831247-22-iscsi devstack@keystone.service[86830]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 09 16:30:52.557549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 16:30:52.557549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: from cryptography.utils import int_from_bytes Aug 09 16:30:52.558350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 16:30:52.558350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: from cryptography.utils import int_from_bytes Aug 09 16:30:52.578135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 16:30:52.578135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: from cryptography.utils import int_from_bytes Aug 09 16:30:52.578711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 09 16:30:52.578711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: from cryptography.utils import int_from_bytes Aug 09 16:30:52.609617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 16:30:52.609791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 16:30:52.610016 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] command line args: [] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 16:30:52.610286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 16:30:52.610464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ================================================================================ {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 16:30:52.610727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] admin_token = **** {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.610973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] config_dir = [] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.611166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.611351 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] config_source = [] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.611532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] debug = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.611731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.612105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.612285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.612535 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.612784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.613120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] list_limit = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.613234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] log_config_append = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.613361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.613538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] log_dir = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.613716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] log_file = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.616881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.617635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.617635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.617831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] publish_errors = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.617964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.618206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.618387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.618633 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.618903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.619139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] standard_threads = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.619374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.619615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.619793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.619971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] use_journal = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.620148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] use_json = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.620323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] use_stderr = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.620499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] use_syslog = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.620731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.620989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.621227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.621486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.621727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.621994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.622236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.622483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.622721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.622985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.623240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.623492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.623759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.623973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.624211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.624459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.624706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.624943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.625197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.625451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.625696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.625934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.626226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.626472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.626725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.626980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.627211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.627369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 09 16:30:52.627537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.627705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 09 16:30:52.627835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.627971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] command line args: [] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 09 16:30:52.628075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.628199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 09 16:30:52.628199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ================================================================================ {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 09 16:30:52.628405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.628405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.628618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] admin_token = **** {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.628731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.628845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] config_dir = [] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.628966 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.629081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.629194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.629312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] config_source = [] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.629432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.629552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] debug = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.629678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.629801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.629987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.630106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.630230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.630368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.630490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.630490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.630710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.630826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.630949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.631064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.631182 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] list_limit = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.631302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.631422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] log_config_append = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.631536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.631656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.631771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.631886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] log_dir = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.631999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.632117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] log_file = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.632230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.632352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.632473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.632586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.632704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.632704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.632921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.633045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.633165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.633287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.633401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.633401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.633616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.633616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.633837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.633972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.634090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.634203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.634406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.634526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.634640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.634759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.634759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.634972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.635097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.635218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.635338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.635539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.635656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.635656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.635876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.635999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.636121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.636236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.636236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.636483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.636603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.79/identity {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.636726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.636841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] publish_errors = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.636958 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.637072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.637193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.637307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.637425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.637539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.637651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.637819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.637967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.638096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.638219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.638335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.638447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] standard_threads = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.638568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.638736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.638854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.639023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.639140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.639254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.639441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.639558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] use_journal = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.639672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.639792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] use_json = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.639906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.640019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] use_stderr = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.640140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.640263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] use_syslog = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.640385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.640499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 09 16:30:52.640612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.640732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.640854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.640854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.641058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.641150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.641306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.641408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.641550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.641687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.641790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.642034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.642143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.642274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.642391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.642495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.642723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.642825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.642955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.643197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.643309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.643428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.643670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.643786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.643908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.644011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.644156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.644267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.644392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.644501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.644641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.644751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.644883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.644992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.645126 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.645228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.645359 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] auth.external = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.645462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.645593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.645701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.645834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.645939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.646113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.646237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.646363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] auth.password = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.646495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.646613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] auth.token = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.646716 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.646834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.646957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.647063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.647168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.647289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.647405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.647517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.647632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.647751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.647874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.647979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.648090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.648203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.648315 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.648427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] credential.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.648548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.648664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.648766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.648885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.649000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.649113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.649234 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.649344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.649457 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.649582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.649705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.649825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.650508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.650640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.650747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.650859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.650973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.651152 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.651269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.651394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.651510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.651623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.651737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.651851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.651971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.652094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.652214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.652330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.652444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.652558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.652677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.652800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.652918 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] federation.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.653031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.653144 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.653264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.653379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.653500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.653615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.653729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.653843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.653970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.654086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.654211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.654333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.654451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.654566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.654680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.654794 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.654915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.655056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.655173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.655286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.655408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.655523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.655645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.655761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.655875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.655990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.656110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.656225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.656350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.656455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.656567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.656681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.656794 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.656981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.657125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.657257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.657382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.657499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.657614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.657729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.657849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.658016 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.658147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.658265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.658380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.658494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.658609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.658723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.658849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.658967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.659082 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.659198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.659312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.659426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.659550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.659666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.659782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.659896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.660011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.660126 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.660248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.660355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.660461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.660559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.660656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.660752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.660855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.660953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661144 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661518 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661579 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661637 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661694 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.661884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.662071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.662146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.662411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.662489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.662637 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.662714 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.662849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.662928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.663089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.663168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.663319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.663388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.663550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.663617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.663781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.663847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.664008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.664076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.664239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.664358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.664475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.664599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.664717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.664832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.664947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.665061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.665206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.665361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.665483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.665638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.666285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.666414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.666538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.666730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.666901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.667088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.667268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.667436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.667601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.667780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] auth.external = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.667950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.668124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.668302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.668469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.668637 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.668805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.668962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.669112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] auth.password = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.669277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.669448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] auth.token = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.669618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.669800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.670014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.670190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.670358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.670525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.670920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.671086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.671248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.671414 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.671578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.671771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.671915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.672034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.672149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] credential.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.672264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.672379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.672494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.672617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.672734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.672849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.672965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.673079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.673193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.673317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.673433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.673548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.673663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.673776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.673892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.674157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.674278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.674393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.674507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.674621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.674736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.674736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.674963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.675078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.675193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.675311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.675427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.675549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.675666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.675781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.675897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.676012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.676126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] federation.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.676248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.676364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.676479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.676593 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.676698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.676812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.676942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.677061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.677175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.677289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.677404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.677518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.677640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.677757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.677872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.678001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.678118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.678249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.678367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] resource.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.678483 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.678598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.678712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.678827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.678950 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.679146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.679290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.679409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.679524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.679524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.679751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.679869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.679984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.680100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.680100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.680315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.680438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.680438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.680654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.680769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.680769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.680985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.681108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.681108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.681327 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] role.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.681442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.681557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] role.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.681671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.681671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.681895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.682056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.682056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.682275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.682389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.682389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.682618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.682740 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.682856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.682970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.682970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.683193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.683311 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.683426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.683547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.683664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.683785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.683909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.684026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.684140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.684254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.684370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.684490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.684619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.684736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.684850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.684965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.685080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.685195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.685322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.685440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.685440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.685655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.685770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.685884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.686023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.686141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.686257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.686377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.686493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.686493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.686718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.686832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.686947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.687062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.687177 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.687299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.687416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.687531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.687645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.687760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.687874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.687997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.688114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.688236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.688351 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.688465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.688581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.688703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.688820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.688934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.689049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.689163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.689276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.689398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.689514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.689629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.689743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.689857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.689997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.690126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.690244 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] token.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.690358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.690473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.690589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.690703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.690826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.690942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.691057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.691171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.691285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.691399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.691522 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.691639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.691754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.691868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.691982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.692116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.692290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.692472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.692657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.692814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.692933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.693050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.693175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.693293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.693408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.693523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.693638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.693755 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.693879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.693879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.694115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.694232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.694232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.694447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.694576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.694576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.694804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.694919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.694919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.695134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.connection = **** {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.695262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.695461 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.695580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.695696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.695811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.695928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.696042 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.696165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.696282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.696282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.696499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.696614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.696730 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.696853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.696971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.697087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.697087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.697302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.697417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.697417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.697643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.697759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] resource.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.697759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.698027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.698146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.698146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.698378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.698494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.698609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.698725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.698725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.698948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.699066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.699066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.699280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.699396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.699396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] role.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.699619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.699735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] role.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.699850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.699964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.700080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.700267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.700394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.700511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.700745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.700863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.700978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.701095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86843) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 16:30:52.701222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.701452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.701814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.702079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.702329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.702580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.702829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.703079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.703507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.703742 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.703993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.704243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.704496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.704750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.704985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.705241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.705626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.705873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.706149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.706359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.706589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.706843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.707101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.707346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.707584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.707823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.708059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.708297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] token.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.708535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.708788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.709028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.709283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.709532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.709769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.710040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.710363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.710567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.710750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.710971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.711209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.711455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.711702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.711932 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.712177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.712987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.713253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.connection = **** {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.713498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.713745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.714001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.714246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.714484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.714720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.714955 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.715189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.715431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.715665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.715907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.716143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.716389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.716623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.716865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.717094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.717335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.717837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 09 16:30:52.718074 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86844) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 09 16:30:52.732756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.732976 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.733196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.743277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.743752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.743985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.744207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.754584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.771889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.772116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.773021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.773255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.826695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.826873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.878594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.878819 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.879369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.879613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.945231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.945429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.945513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.945664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.945784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.945865 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.945939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.946112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.946196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.946327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.946411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.946543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.973837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.974129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.974216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.974320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.974509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.974727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.974942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.975156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.975278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.975509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.975741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.975963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.976178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.976395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.987361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:52.992441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.000642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.000980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.001201 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.001407 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.001630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.001849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.002097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86844) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.003684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.003911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.004134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.004354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.004574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.004790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.005005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86843) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 09 16:30:53.020052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.020203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.020285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 16:30:53.020350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 16:30:53.020594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.020709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.020788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 16:30:53.020862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 09 16:30:53.021057 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.021140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.021318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.021403 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.021574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.021657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.021830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.021902 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.033849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.034079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.034162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.034301 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.034414 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.034574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.034700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.034857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.041888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.042068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.044999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.045094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.045287 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.045378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.045569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.045652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.045846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.045928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.046149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.046231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.046413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.046505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.046679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.046759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.046947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.047025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.047216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.047298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.047491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.047580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.047756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.047836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.070691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.071693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.072715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.073074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.073741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.074052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.077843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.078095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.078382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.078887 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.079395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.079395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.085348 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.085638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.086930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.087220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.089803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.090122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.090419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.090714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.090998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.091778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.091903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.092058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.092194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.092326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.092487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.093033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.093358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.093596 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.093863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.094170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.108522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.108802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.110321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.110606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.112793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.114672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.115962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.116261 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.117851 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.118179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.120805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.125190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.125294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.126031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.127213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.127513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.127805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.128103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.128383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.128671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.129320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.129627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.130008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.130247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.130529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.130809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.145335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.145634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.145907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.146196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.147956 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.148261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.148532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.148797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.153760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.155375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.155656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.156362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.157982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.158252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.158515 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.158940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.159087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.159391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.159670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.161190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.161478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.161780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.162100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.162385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.171187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.171479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.171760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.172038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.172315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.172592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.172868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.174539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.174834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.175116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.175393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.175674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.175954 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.176224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.190415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.193402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.193816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.196409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.196928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.198588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.200094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.200394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.200512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.200675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.202098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.203706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.203995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.204277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.207175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.208808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.211049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.211936 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.212254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.212583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.212850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.213053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.215840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.216131 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.216420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.216695 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.221295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.221595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.221876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.225417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.225727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.226050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.228800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=86843) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.229087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.229369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.229646 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.229918 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.230229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.230505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.230771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.231152 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.231439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.231712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.231977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.232252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.232513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86843) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.233236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=86844) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 09 16:30:53.233521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.233801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.234105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.234394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.234881 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.235129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.235407 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.235789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.236097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.236379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.236652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.236920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.237182 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86844) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 09 16:30:53.266932 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d9d6a379c0 pid: 86843 (default app) Aug 09 16:30:53.270398 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191498b-5c67-42c5-a545-78c512c1a279 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:30:53.270682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191498b-5c67-42c5-a545-78c512c1a279 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:30:53.270953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191498b-5c67-42c5-a545-78c512c1a279 None None] PATH_INFO: `/v3` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:30:53.272194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d9d6a379c0 pid: 86844 (default app) Aug 09 16:30:53.275315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1/1] 10.222.0.79 () {50 vars in 791 bytes} [Tue Aug 9 16:30:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:02.291539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f3b0a-6e5e-4b7e-a76a-a405b5e68cb3 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:02.292105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f3b0a-6e5e-4b7e-a76a-a405b5e68cb3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:02.292105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f3b0a-6e5e-4b7e-a76a-a405b5e68cb3 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:02.296561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1/2] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:02.301906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:02.302343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:02.302481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:02.372838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: INFO dbcounter [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] Registered counter for database keystone Aug 09 16:31:02.394631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG oslo_db.sqlalchemy.engines [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86843) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 16:31:02.401012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writer thread running {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 16:31:02.579890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG passlib.handlers.bcrypt [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86843) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 16:31:02.580224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG passlib.handlers.bcrypt [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86843) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 16:31:02.635170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:02.636752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:02.766623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 2/3] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 09 16:31:02.912379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f3b0a-6e5e-4b7e-a76a-a405b5e68cb3 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:02.982271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: INFO dbcounter [None req-b12f3b0a-6e5e-4b7e-a76a-a405b5e68cb3 None None] Registered counter for database keystone Aug 09 16:31:03.001186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG oslo_db.sqlalchemy.engines [None req-b12f3b0a-6e5e-4b7e-a76a-a405b5e68cb3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86844) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 09 16:31:03.007517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writer thread running {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 09 16:31:03.126045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f3b0a-6e5e-4b7e-a76a-a405b5e68cb3 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:03.130331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c07f6-f104-450f-9dff-10089123cb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:03.130949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c07f6-f104-450f-9dff-10089123cb20 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:03.131228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c07f6-f104-450f-9dff-10089123cb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:03.131498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c07f6-f104-450f-9dff-10089123cb20 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:03.139739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c07f6-f104-450f-9dff-10089123cb20 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:03.191048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: WARNING keystone.common.rbac_enforcer.enforcer [None req-528c07f6-f104-450f-9dff-10089123cb20 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 16:31:03.191317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c07f6-f104-450f-9dff-10089123cb20 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:03.201430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 2/4] 10.222.0.79 () {58 vars in 1324 bytes} [Tue Aug 9 16:31:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 16:31:03.207369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:03.222511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497abbc6-5b6c-40bf-bce8-f8933cc0cd40 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:03.226924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe98e2-9d4a-48cb-afd7-d062e26e68d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:03.228566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe98e2-9d4a-48cb-afd7-d062e26e68d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:03.228845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe98e2-9d4a-48cb-afd7-d062e26e68d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:03.229120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe98e2-9d4a-48cb-afd7-d062e26e68d8 admin admin] PATH_INFO: `/v3/projects/e0a53b8ab06c430ea7305247e9df6f92` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:03.230608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe98e2-9d4a-48cb-afd7-d062e26e68d8 admin admin] RBAC: Authorizing `identity:get_project(project_id=e0a53b8ab06c430ea7305247e9df6f92)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:03.280008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: WARNING keystone.common.rbac_enforcer.enforcer [None req-46fe98e2-9d4a-48cb-afd7-d062e26e68d8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 09 16:31:03.280347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe98e2-9d4a-48cb-afd7-d062e26e68d8 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:03.284928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 3/5] 10.222.0.79 () {56 vars in 1207 bytes} [Tue Aug 9 16:31:03 2022] GET /identity/v3/projects/e0a53b8ab06c430ea7305247e9df6f92 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:05.263153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30195f97-ebcc-425b-9a37-163dc3baedb2 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:05.263769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30195f97-ebcc-425b-9a37-163dc3baedb2 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:05.263769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30195f97-ebcc-425b-9a37-163dc3baedb2 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:05.267774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 3/6] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:05.273255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82ab26-a4b7-4827-86b5-7eff374b9835 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:05.273677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82ab26-a4b7-4827-86b5-7eff374b9835 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:05.273795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82ab26-a4b7-4827-86b5-7eff374b9835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:05.294758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-0b82ab26-a4b7-4827-86b5-7eff374b9835 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:05.296271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0b82ab26-a4b7-4827-86b5-7eff374b9835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:05.304055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 4/7] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 09 16:31:05.326675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30195f97-ebcc-425b-9a37-163dc3baedb2 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:05.338627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30195f97-ebcc-425b-9a37-163dc3baedb2 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:05.343081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32280675-8eb9-4d15-8d08-fdb3fc89dd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:05.343692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32280675-8eb9-4d15-8d08-fdb3fc89dd14 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:05.343971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32280675-8eb9-4d15-8d08-fdb3fc89dd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:05.344237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32280675-8eb9-4d15-8d08-fdb3fc89dd14 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:05.351972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32280675-8eb9-4d15-8d08-fdb3fc89dd14 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:05.353724 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32280675-8eb9-4d15-8d08-fdb3fc89dd14 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:05.362511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 4/8] 10.222.0.79 () {58 vars in 1324 bytes} [Tue Aug 9 16:31:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 16:31:05.368418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82ab26-a4b7-4827-86b5-7eff374b9835 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:05.377480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82ab26-a4b7-4827-86b5-7eff374b9835 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:05.381801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cda02-ce48-43d6-91b2-dbc53551cf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:05.382441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cda02-ce48-43d6-91b2-dbc53551cf61 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:05.382714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cda02-ce48-43d6-91b2-dbc53551cf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:05.382982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cda02-ce48-43d6-91b2-dbc53551cf61 admin admin] PATH_INFO: `/v3/users/ea8ba450a8204c87830e1b4dce2332ee` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:05.384314 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cda02-ce48-43d6-91b2-dbc53551cf61 admin admin] RBAC: Authorizing `identity:get_user(user_id=ea8ba450a8204c87830e1b4dce2332ee)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:05.386683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cda02-ce48-43d6-91b2-dbc53551cf61 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:05.388174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 5/9] 10.222.0.79 () {56 vars in 1198 bytes} [Tue Aug 9 16:31:05 2022] GET /identity/v3/users/ea8ba450a8204c87830e1b4dce2332ee => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:07.474519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31820e44-e51c-46d0-8fae-4565cfacc285 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:07.475147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31820e44-e51c-46d0-8fae-4565cfacc285 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:07.475147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31820e44-e51c-46d0-8fae-4565cfacc285 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:07.476384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 5/10] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:07.481650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8381d14-d2a5-4b70-8931-e4a3cd682e54 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:07.482124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8381d14-d2a5-4b70-8931-e4a3cd682e54 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:07.482216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8381d14-d2a5-4b70-8931-e4a3cd682e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:07.493658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5078d50-83e3-4be5-a019-6f41ffabe0d9 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:07.493934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5078d50-83e3-4be5-a019-6f41ffabe0d9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:07.494227 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5078d50-83e3-4be5-a019-6f41ffabe0d9 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:07.495529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 6/11] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:31:07.500483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-e8381d14-d2a5-4b70-8931-e4a3cd682e54 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:07.500597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed573d4-015f-46d2-b921-7346e42aecee None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:07.500830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed573d4-015f-46d2-b921-7346e42aecee None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:07.501105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed573d4-015f-46d2-b921-7346e42aecee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:07.501642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e8381d14-d2a5-4b70-8931-e4a3cd682e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:07.544833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG passlib.handlers.bcrypt [None req-fed573d4-015f-46d2-b921-7346e42aecee None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86844) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 09 16:31:07.545254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG passlib.handlers.bcrypt [None req-fed573d4-015f-46d2-b921-7346e42aecee None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86844) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 09 16:31:07.585299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 6/12] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:07.591822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-fed573d4-015f-46d2-b921-7346e42aecee None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:07.593445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fed573d4-015f-46d2-b921-7346e42aecee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:07.608120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8381d14-d2a5-4b70-8931-e4a3cd682e54 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:07.632639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:07.693455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 7/13] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:07.715345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed573d4-015f-46d2-b921-7346e42aecee None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:07.715874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8381d14-d2a5-4b70-8931-e4a3cd682e54 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:07.750357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17a72c-4016-4143-8e1c-564a802a25a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:07.750897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17a72c-4016-4143-8e1c-564a802a25a8 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:07.751175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17a72c-4016-4143-8e1c-564a802a25a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:07.751442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17a72c-4016-4143-8e1c-564a802a25a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:07.763131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17a72c-4016-4143-8e1c-564a802a25a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-7e17a72c-4016-4143-8e1c-564a802a25a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:07.771808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:07.772747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:07.773636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 7/14] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:31:07.778933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17a72c-4016-4143-8e1c-564a802a25a8 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:07.822220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed573d4-015f-46d2-b921-7346e42aecee None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:07.824440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:07.860211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da48aa-8293-4e1b-89e9-c74671f5e9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:07.860874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:07.861054 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da48aa-8293-4e1b-89e9-c74671f5e9ee None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:07.861329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da48aa-8293-4e1b-89e9-c74671f5e9ee None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:07.861601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da48aa-8293-4e1b-89e9-c74671f5e9ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:07.880122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17a72c-4016-4143-8e1c-564a802a25a8 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:07.913693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42005f17-242b-4731-8d2d-4f0b8e1b9164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:07.914836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42005f17-242b-4731-8d2d-4f0b8e1b9164 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:07.915116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42005f17-242b-4731-8d2d-4f0b8e1b9164 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:07.915387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42005f17-242b-4731-8d2d-4f0b8e1b9164 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:07.916073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42005f17-242b-4731-8d2d-4f0b8e1b9164 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:07.916372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42005f17-242b-4731-8d2d-4f0b8e1b9164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:07.917765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42005f17-242b-4731-8d2d-4f0b8e1b9164 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:07.929457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 8/15] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:07.929806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da48aa-8293-4e1b-89e9-c74671f5e9ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:07.931846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da48aa-8293-4e1b-89e9-c74671f5e9ee None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:07.936148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42005f17-242b-4731-8d2d-4f0b8e1b9164 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:08.033383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 8/16] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 16:31:08.039808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da48aa-8293-4e1b-89e9-c74671f5e9ee None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:08.043068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:08.145797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da48aa-8293-4e1b-89e9-c74671f5e9ee None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:08.146039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42005f17-242b-4731-8d2d-4f0b8e1b9164 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:08.178795 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97598526-1cdf-41bb-a8b9-6d593c6f68d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:08.179358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9cd0e-1223-45a8-ad33-122b56fe2cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:08.179626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97598526-1cdf-41bb-a8b9-6d593c6f68d2 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:08.180621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9cd0e-1223-45a8-ad33-122b56fe2cc4 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:08.180746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97598526-1cdf-41bb-a8b9-6d593c6f68d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:08.180826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9cd0e-1223-45a8-ad33-122b56fe2cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:08.180974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97598526-1cdf-41bb-a8b9-6d593c6f68d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:08.181042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9cd0e-1223-45a8-ad33-122b56fe2cc4 None admin] PATH_INFO: `/v3/users/ea8ba450a8204c87830e1b4dce2332ee` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:08.182221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9cd0e-1223-45a8-ad33-122b56fe2cc4 None admin] RBAC: Authorizing `identity:get_user(user_id=ea8ba450a8204c87830e1b4dce2332ee)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:08.183695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9cd0e-1223-45a8-ad33-122b56fe2cc4 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:08.185268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 9/17] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:07 2022] GET /identity/v3/users/ea8ba450a8204c87830e1b4dce2332ee => generated 247 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:08.190801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9cd0e-1223-45a8-ad33-122b56fe2cc4 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:08.199383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97598526-1cdf-41bb-a8b9-6d593c6f68d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:08.200873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97598526-1cdf-41bb-a8b9-6d593c6f68d2 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:08.205445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-97598526-1cdf-41bb-a8b9-6d593c6f68d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:31:08.219721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:08.220641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:31:08.221448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:31:08.221448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:31:08.221448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:08.221448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 9/18] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:31:08.227384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97598526-1cdf-41bb-a8b9-6d593c6f68d2 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:08.282653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:08.293729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9cd0e-1223-45a8-ad33-122b56fe2cc4 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:08.325654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97598526-1cdf-41bb-a8b9-6d593c6f68d2 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:08.327624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046a8b8-6f5f-405b-a7c2-9c7d8298b7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:08.328255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a8b8-6f5f-405b-a7c2-9c7d8298b7d8 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:08.328532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a8b8-6f5f-405b-a7c2-9c7d8298b7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:08.328798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a8b8-6f5f-405b-a7c2-9c7d8298b7d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:08.329911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046a8b8-6f5f-405b-a7c2-9c7d8298b7d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:08.331357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046a8b8-6f5f-405b-a7c2-9c7d8298b7d8 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:08.332936 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 10/19] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:08.339354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046a8b8-6f5f-405b-a7c2-9c7d8298b7d8 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:08.359931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e618bb-47e4-442c-be2d-c20ed972e0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:08.360579 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e618bb-47e4-442c-be2d-c20ed972e0b3 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:08.360849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e618bb-47e4-442c-be2d-c20ed972e0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:08.361116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e618bb-47e4-442c-be2d-c20ed972e0b3 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:08.361743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e618bb-47e4-442c-be2d-c20ed972e0b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:08.362070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e618bb-47e4-442c-be2d-c20ed972e0b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:08.363434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e618bb-47e4-442c-be2d-c20ed972e0b3 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:08.380634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 10/20] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:08.424085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:08.444747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046a8b8-6f5f-405b-a7c2-9c7d8298b7d8 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:08.482625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e121219-57ce-4c3f-905b-deb93b721a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:08.483407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e121219-57ce-4c3f-905b-deb93b721a37 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:08.483495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e121219-57ce-4c3f-905b-deb93b721a37 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:08.483749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e121219-57ce-4c3f-905b-deb93b721a37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:08.484483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e121219-57ce-4c3f-905b-deb93b721a37 None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.domain.id=default, user.id=ea8ba450a8204c87830e1b4dce2332ee) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:08.484783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e121219-57ce-4c3f-905b-deb93b721a37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:08.486288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e121219-57ce-4c3f-905b-deb93b721a37 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:08.493718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 11/21] 10.222.0.79 () {56 vars in 1322 bytes} [Tue Aug 9 16:31:08 2022] GET /identity/v3/role_assignments?user.id=ea8ba450a8204c87830e1b4dce2332ee&scope.domain.id=default&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:10.848322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8898-b7ad-49fc-876d-b3860e949f0f None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:10.849283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8898-b7ad-49fc-876d-b3860e949f0f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:10.849769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8898-b7ad-49fc-876d-b3860e949f0f None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:10.851664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 11/22] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:10.857408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac1e28-eb16-4c34-89c5-c1f12a35eba2 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:10.858393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac1e28-eb16-4c34-89c5-c1f12a35eba2 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:10.858892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac1e28-eb16-4c34-89c5-c1f12a35eba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:10.887061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-3cac1e28-eb16-4c34-89c5-c1f12a35eba2 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:10.888713 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3cac1e28-eb16-4c34-89c5-c1f12a35eba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:10.980588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 12/23] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:11.021811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e8898-b7ad-49fc-876d-b3860e949f0f None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:11.077830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f4b6-069e-478d-945c-e1ccdd8b7136 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.078495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f4b6-069e-478d-945c-e1ccdd8b7136 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.078985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f4b6-069e-478d-945c-e1ccdd8b7136 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.080502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 13/24] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:11.085720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212839-5cd4-45ca-bd12-5b6206d4fabd None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.086788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212839-5cd4-45ca-bd12-5b6206d4fabd None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.087253 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212839-5cd4-45ca-bd12-5b6206d4fabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.123058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a0212839-5cd4-45ca-bd12-5b6206d4fabd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:11.124756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a0212839-5cd4-45ca-bd12-5b6206d4fabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:11.158489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e8898-b7ad-49fc-876d-b3860e949f0f None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:11.189694 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:11.227451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf648b-e790-4389-98c7-e05a1af9faf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:11.228122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf648b-e790-4389-98c7-e05a1af9faf6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.228395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf648b-e790-4389-98c7-e05a1af9faf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.228715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf648b-e790-4389-98c7-e05a1af9faf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.238007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 14/25] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:11.239265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530b7de-1cb9-4296-b332-d03657d81360 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.239548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530b7de-1cb9-4296-b332-d03657d81360 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.239816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530b7de-1cb9-4296-b332-d03657d81360 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.244457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 15/26] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:11.250003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8b9aa-7a44-43f4-b950-b1f2117d8cbe None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.250287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8b9aa-7a44-43f4-b950-b1f2117d8cbe None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.250567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8b9aa-7a44-43f4-b950-b1f2117d8cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.255917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf648b-e790-4389-98c7-e05a1af9faf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:11.257609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf648b-e790-4389-98c7-e05a1af9faf6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:11.268556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:11.287010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-99a8b9aa-7a44-43f4-b950-b1f2117d8cbe None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:11.288208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-99a8b9aa-7a44-43f4-b950-b1f2117d8cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-94bf648b-e790-4389-98c7-e05a1af9faf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:31:11.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:11.290926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:31:11.292145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:31:11.292145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:31:11.292145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:11.292145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 12/27] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:11.292480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf648b-e790-4389-98c7-e05a1af9faf6 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:11.413913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 16/28] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:11.415985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8b9aa-7a44-43f4-b950-b1f2117d8cbe None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:11.468542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf648b-e790-4389-98c7-e05a1af9faf6 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:11.561719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8b9aa-7a44-43f4-b950-b1f2117d8cbe None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:11.602685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1a8f2-ef58-4dca-b37e-4222bd0ecf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:11.603524 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1a8f2-ef58-4dca-b37e-4222bd0ecf9d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.603939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1a8f2-ef58-4dca-b37e-4222bd0ecf9d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.604340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1a8f2-ef58-4dca-b37e-4222bd0ecf9d None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.605301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1a8f2-ef58-4dca-b37e-4222bd0ecf9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:11.610220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1a8f2-ef58-4dca-b37e-4222bd0ecf9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:11.610220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1a8f2-ef58-4dca-b37e-4222bd0ecf9d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:11.613716 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d189157-0670-47bb-b446-0cdc9903680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:11.614553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d189157-0670-47bb-b446-0cdc9903680c None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.615007 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d189157-0670-47bb-b446-0cdc9903680c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.615466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d189157-0670-47bb-b446-0cdc9903680c None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.616294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d189157-0670-47bb-b446-0cdc9903680c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:11.618500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d189157-0670-47bb-b446-0cdc9903680c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:11.619537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:11.628550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 17/29] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:11.641598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c235bf9-a381-4776-a7f4-8fc34d32757f None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.641598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c235bf9-a381-4776-a7f4-8fc34d32757f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.641598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c235bf9-a381-4776-a7f4-8fc34d32757f None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.641598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 18/30] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:11.641598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c235bf9-a381-4776-a7f4-8fc34d32757f None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:11.741579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c235bf9-a381-4776-a7f4-8fc34d32757f None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:11.779176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a281c47-444f-4dcd-90c4-456e01b0a982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:11.779852 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a281c47-444f-4dcd-90c4-456e01b0a982 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.780131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a281c47-444f-4dcd-90c4-456e01b0a982 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.780447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a281c47-444f-4dcd-90c4-456e01b0a982 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.792606 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a281c47-444f-4dcd-90c4-456e01b0a982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-7a281c47-444f-4dcd-90c4-456e01b0a982 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:11.797612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:11.798700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:11.799475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:11.803589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 19/31] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:31:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:11.804962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a281c47-444f-4dcd-90c4-456e01b0a982 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:11.863652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:11.879772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 13/32] 10.222.0.79 () {60 vars in 1131 bytes} [Tue Aug 9 16:31:11 2022] POST /identity/v3/roles => generated 225 bytes in 587 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:11.882037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00fe5c-01ee-44cf-ab3b-da3e57cd224c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.882577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00fe5c-01ee-44cf-ab3b-da3e57cd224c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.883055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00fe5c-01ee-44cf-ab3b-da3e57cd224c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.904881 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-fc00fe5c-01ee-44cf-ab3b-da3e57cd224c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:11.907078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fc00fe5c-01ee-44cf-ab3b-da3e57cd224c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:11.919023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a281c47-444f-4dcd-90c4-456e01b0a982 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:11.938364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:11.970421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28436bf-ef05-487f-8d1a-0e3403a34d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:11.971058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28436bf-ef05-487f-8d1a-0e3403a34d10 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:11.971403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28436bf-ef05-487f-8d1a-0e3403a34d10 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:11.971678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28436bf-ef05-487f-8d1a-0e3403a34d10 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:11.973311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28436bf-ef05-487f-8d1a-0e3403a34d10 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:11.975540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28436bf-ef05-487f-8d1a-0e3403a34d10 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:11.999083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 14/33] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:12.000373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00fe5c-01ee-44cf-ab3b-da3e57cd224c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:12.123166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00fe5c-01ee-44cf-ab3b-da3e57cd224c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:12.160949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cecf8-26f9-419b-8108-84a909faa761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:12.162031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cecf8-26f9-419b-8108-84a909faa761 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.162529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cecf8-26f9-419b-8108-84a909faa761 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.162999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cecf8-26f9-419b-8108-84a909faa761 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.164465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cecf8-26f9-419b-8108-84a909faa761 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:12.165029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cecf8-26f9-419b-8108-84a909faa761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:12.166763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cecf8-26f9-419b-8108-84a909faa761 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:12.179672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 15/34] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:12.181212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c40cc4-73b5-49ac-9b38-1c804d8c81e5 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.181803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c40cc4-73b5-49ac-9b38-1c804d8c81e5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.182297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c40cc4-73b5-49ac-9b38-1c804d8c81e5 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.183723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 16/35] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:12.184626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c40cc4-73b5-49ac-9b38-1c804d8c81e5 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:12.208323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:12.214397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 20/36] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:31:11 2022] POST /identity/v3/projects => generated 301 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:12.215397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28436bf-ef05-487f-8d1a-0e3403a34d10 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:12.331381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c40cc4-73b5-49ac-9b38-1c804d8c81e5 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:12.365423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28436bf-ef05-487f-8d1a-0e3403a34d10 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:12.376093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:12.383380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a44f4-54d9-4ed1-83df-5cf9ceab9705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:12.384288 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a44f4-54d9-4ed1-83df-5cf9ceab9705 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.388455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a44f4-54d9-4ed1-83df-5cf9ceab9705 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.388455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a44f4-54d9-4ed1-83df-5cf9ceab9705 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.388455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a44f4-54d9-4ed1-83df-5cf9ceab9705 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:12.388455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a44f4-54d9-4ed1-83df-5cf9ceab9705 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:12.408858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf069d6-6c49-41b9-adf9-e3d771c7ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:12.409790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf069d6-6c49-41b9-adf9-e3d771c7ab12 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.411504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf069d6-6c49-41b9-adf9-e3d771c7ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.411504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf069d6-6c49-41b9-adf9-e3d771c7ab12 None admin] PATH_INFO: `/v3/users/ea8ba450a8204c87830e1b4dce2332ee` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.411699 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf069d6-6c49-41b9-adf9-e3d771c7ab12 None admin] RBAC: Authorizing `identity:get_user(user_id=ea8ba450a8204c87830e1b4dce2332ee)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:12.413196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf069d6-6c49-41b9-adf9-e3d771c7ab12 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:12.415081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 21/37] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:12 2022] GET /identity/v3/users/ea8ba450a8204c87830e1b4dce2332ee => generated 247 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:12.416147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b37116-2be0-4e0b-af15-968e15170b47 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.416363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b37116-2be0-4e0b-af15-968e15170b47 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.416639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b37116-2be0-4e0b-af15-968e15170b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.435816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-00b37116-2be0-4e0b-af15-968e15170b47 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:12.437348 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-00b37116-2be0-4e0b-af15-968e15170b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:12.466258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 17/38] 10.222.0.79 () {60 vars in 1131 bytes} [Tue Aug 9 16:31:12 2022] POST /identity/v3/roles => generated 219 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:12.467179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a44f4-54d9-4ed1-83df-5cf9ceab9705 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:12.518246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 22/39] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:12.519559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba93a45-ccfe-4b88-9806-7994340d9b43 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.519842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba93a45-ccfe-4b88-9806-7994340d9b43 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.520112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba93a45-ccfe-4b88-9806-7994340d9b43 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.521492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 23/40] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:12.532290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ee0dc-9a8e-424b-a093-9dadd317a095 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.532679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ee0dc-9a8e-424b-a093-9dadd317a095 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.533056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ee0dc-9a8e-424b-a093-9dadd317a095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.552487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-0f2ee0dc-9a8e-424b-a093-9dadd317a095 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:12.553734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0f2ee0dc-9a8e-424b-a093-9dadd317a095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:12.591005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:12.613898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a44f4-54d9-4ed1-83df-5cf9ceab9705 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:12.630811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 24/41] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:12.635135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ee0dc-9a8e-424b-a093-9dadd317a095 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:12.660962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543b83a-7fc2-4fdb-b19f-7a7e34480e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:12.662166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543b83a-7fc2-4fdb-b19f-7a7e34480e5f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.662299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543b83a-7fc2-4fdb-b19f-7a7e34480e5f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.662571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543b83a-7fc2-4fdb-b19f-7a7e34480e5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.664395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543b83a-7fc2-4fdb-b19f-7a7e34480e5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:12.666980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543b83a-7fc2-4fdb-b19f-7a7e34480e5f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:12.668960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 18/42] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:12.670120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543b83a-7fc2-4fdb-b19f-7a7e34480e5f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:12.673579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:12.743934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ee0dc-9a8e-424b-a093-9dadd317a095 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:12.775274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543b83a-7fc2-4fdb-b19f-7a7e34480e5f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:12.778573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc597fd7-959b-49bb-9cab-d1f9bd3f96a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:12.779178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc597fd7-959b-49bb-9cab-d1f9bd3f96a3 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.779459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc597fd7-959b-49bb-9cab-d1f9bd3f96a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.779726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc597fd7-959b-49bb-9cab-d1f9bd3f96a3 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.780392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc597fd7-959b-49bb-9cab-d1f9bd3f96a3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:12.781778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc597fd7-959b-49bb-9cab-d1f9bd3f96a3 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:12.810441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a47fb-d24a-4d9d-85c7-de6b070354d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:12.811063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a47fb-d24a-4d9d-85c7-de6b070354d4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.811341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a47fb-d24a-4d9d-85c7-de6b070354d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.811608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a47fb-d24a-4d9d-85c7-de6b070354d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.812766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a47fb-d24a-4d9d-85c7-de6b070354d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:12.814232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a47fb-d24a-4d9d-85c7-de6b070354d4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:12.815816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 19/43] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:12.816625 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a47fb-d24a-4d9d-85c7-de6b070354d4 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:12.835224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:12.859618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 25/44] 10.222.0.79 () {60 vars in 1131 bytes} [Tue Aug 9 16:31:12 2022] POST /identity/v3/roles => generated 223 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:12.859937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc597fd7-959b-49bb-9cab-d1f9bd3f96a3 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:12.939919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a47fb-d24a-4d9d-85c7-de6b070354d4 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:12.973835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:12.979576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1668a-5f9e-48ac-8c87-1b490701f68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:12.980255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1668a-5f9e-48ac-8c87-1b490701f68e None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:12.980538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1668a-5f9e-48ac-8c87-1b490701f68e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:12.980814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1668a-5f9e-48ac-8c87-1b490701f68e None admin] PATH_INFO: `/v3/domains/default/users/ea8ba450a8204c87830e1b4dce2332ee/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:12.981390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc597fd7-959b-49bb-9cab-d1f9bd3f96a3 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:12.983353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1668a-5f9e-48ac-8c87-1b490701f68e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ea8ba450a8204c87830e1b4dce2332ee, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:12.985568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1668a-5f9e-48ac-8c87-1b490701f68e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:13.020263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3cd5f7-4c73-4730-8f03-2c78cd6fdd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:13.020880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cd5f7-4c73-4730-8f03-2c78cd6fdd0a None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:13.021158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cd5f7-4c73-4730-8f03-2c78cd6fdd0a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:13.021434 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3cd5f7-4c73-4730-8f03-2c78cd6fdd0a None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:13.022124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3cd5f7-4c73-4730-8f03-2c78cd6fdd0a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:13.023630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3cd5f7-4c73-4730-8f03-2c78cd6fdd0a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:13.052415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 20/45] 10.222.0.79 () {58 vars in 1361 bytes} [Tue Aug 9 16:31:12 2022] PUT /identity/v3/domains/default/users/ea8ba450a8204c87830e1b4dce2332ee/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:31:13.134776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 26/46] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:31:12 2022] POST /identity/v3/projects => generated 312 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:15.056000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394203-d7c6-42de-81b7-b8497c81c714 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.056544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394203-d7c6-42de-81b7-b8497c81c714 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.056544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45394203-d7c6-42de-81b7-b8497c81c714 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.057879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 21/47] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:15.063531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cfeb7-520f-4ad4-abab-32957c04c1f8 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.063997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cfeb7-520f-4ad4-abab-32957c04c1f8 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.064200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8cfeb7-520f-4ad4-abab-32957c04c1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.082502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8d8cfeb7-520f-4ad4-abab-32957c04c1f8 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:15.083771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8d8cfeb7-520f-4ad4-abab-32957c04c1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:15.151224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 27/48] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:15.173939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394203-d7c6-42de-81b7-b8497c81c714 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:15.213231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37846f1-52a6-467b-a7f0-18af18c11f6e None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.213509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37846f1-52a6-467b-a7f0-18af18c11f6e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.213779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37846f1-52a6-467b-a7f0-18af18c11f6e None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.215152 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 28/49] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:31:15.220298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137d685-7cc4-4310-a442-70e228555c40 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.220578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137d685-7cc4-4310-a442-70e228555c40 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.220859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137d685-7cc4-4310-a442-70e228555c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.238139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d137d685-7cc4-4310-a442-70e228555c40 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:15.239340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d137d685-7cc4-4310-a442-70e228555c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:15.253162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:15.277782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45394203-d7c6-42de-81b7-b8497c81c714 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:15.309530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 29/50] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:15.312296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf0f60-26ba-4e9c-b570-a6e53be37f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:15.312924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf0f60-26ba-4e9c-b570-a6e53be37f2c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.313204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf0f60-26ba-4e9c-b570-a6e53be37f2c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.313476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf0f60-26ba-4e9c-b570-a6e53be37f2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.325747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf0f60-26ba-4e9c-b570-a6e53be37f2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-1adf0f60-26ba-4e9c-b570-a6e53be37f2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:15.331738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:15.332754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:15.333514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:15.334128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137d685-7cc4-4310-a442-70e228555c40 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:15.334199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 22/51] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:31:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:15.340050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf0f60-26ba-4e9c-b570-a6e53be37f2c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:15.340942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:15.439558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137d685-7cc4-4310-a442-70e228555c40 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:15.451423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf0f60-26ba-4e9c-b570-a6e53be37f2c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:15.474673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc92f34-f054-4228-989a-f4568b088294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:15.475295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92f34-f054-4228-989a-f4568b088294 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.475572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92f34-f054-4228-989a-f4568b088294 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.476359 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc92f34-f054-4228-989a-f4568b088294 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.477477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc92f34-f054-4228-989a-f4568b088294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:15.478946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc92f34-f054-4228-989a-f4568b088294 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:15.480686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 30/52] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:15.486170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc92f34-f054-4228-989a-f4568b088294 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:15.486420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f091c0-a2c2-40ef-807a-fd769536c6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:15.487034 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f091c0-a2c2-40ef-807a-fd769536c6fa None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.487312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f091c0-a2c2-40ef-807a-fd769536c6fa None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.487585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f091c0-a2c2-40ef-807a-fd769536c6fa None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.488248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f091c0-a2c2-40ef-807a-fd769536c6fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:15.488547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f091c0-a2c2-40ef-807a-fd769536c6fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:15.489971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f091c0-a2c2-40ef-807a-fd769536c6fa None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:15.492879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:15.500044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 23/53] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:15.505348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f091c0-a2c2-40ef-807a-fd769536c6fa None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:15.564489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:15.595334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc92f34-f054-4228-989a-f4568b088294 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:15.610022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f091c0-a2c2-40ef-807a-fd769536c6fa None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:15.630144 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98b035-319a-4605-b535-1c1f696f9487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:15.630752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98b035-319a-4605-b535-1c1f696f9487 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.631033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98b035-319a-4605-b535-1c1f696f9487 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.631317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98b035-319a-4605-b535-1c1f696f9487 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.631960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98b035-319a-4605-b535-1c1f696f9487 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:15.633906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98b035-319a-4605-b535-1c1f696f9487 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:15.645408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a543063-a380-49a1-8abd-1fd24712df6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:15.646068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a543063-a380-49a1-8abd-1fd24712df6e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.646349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a543063-a380-49a1-8abd-1fd24712df6e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.646619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a543063-a380-49a1-8abd-1fd24712df6e None admin] PATH_INFO: `/v3/users/ea8ba450a8204c87830e1b4dce2332ee` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.648364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a543063-a380-49a1-8abd-1fd24712df6e None admin] RBAC: Authorizing `identity:get_user(user_id=ea8ba450a8204c87830e1b4dce2332ee)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:15.649860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a543063-a380-49a1-8abd-1fd24712df6e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:15.651543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 24/54] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:15 2022] GET /identity/v3/users/ea8ba450a8204c87830e1b4dce2332ee => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:15.657018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a543063-a380-49a1-8abd-1fd24712df6e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:15.722040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:15.756609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a543063-a380-49a1-8abd-1fd24712df6e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:15.790700 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f22ba-79aa-4156-890d-c3d249c8831b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:15.791318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f22ba-79aa-4156-890d-c3d249c8831b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.791592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f22ba-79aa-4156-890d-c3d249c8831b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.791857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f22ba-79aa-4156-890d-c3d249c8831b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.792954 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f22ba-79aa-4156-890d-c3d249c8831b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:15.794425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f22ba-79aa-4156-890d-c3d249c8831b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:15.795961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 25/55] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:15.801713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f22ba-79aa-4156-890d-c3d249c8831b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:15.826827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 31/56] 10.222.0.79 () {60 vars in 1140 bytes} [Tue Aug 9 16:31:15 2022] POST /identity/v3/projects => generated 298 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:31:15.910844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f22ba-79aa-4156-890d-c3d249c8831b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:15.947284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a688b3-a58b-4ebb-a9ae-132762787dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:15.947901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a688b3-a58b-4ebb-a9ae-132762787dc5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:15.948172 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a688b3-a58b-4ebb-a9ae-132762787dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:15.948469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a688b3-a58b-4ebb-a9ae-132762787dc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:15.949228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a688b3-a58b-4ebb-a9ae-132762787dc5 None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.domain.id=default, user.id=ea8ba450a8204c87830e1b4dce2332ee) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:15.949525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a688b3-a58b-4ebb-a9ae-132762787dc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:15.951012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a688b3-a58b-4ebb-a9ae-132762787dc5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:15.953453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:15.965859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 26/57] 10.222.0.79 () {56 vars in 1322 bytes} [Tue Aug 9 16:31:15 2022] GET /identity/v3/role_assignments?user.id=ea8ba450a8204c87830e1b4dce2332ee&scope.domain.id=default&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:17.867901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee1a24-7f71-4b7c-bfd4-389ad489ac4a None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:17.868455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee1a24-7f71-4b7c-bfd4-389ad489ac4a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:17.868455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee1a24-7f71-4b7c-bfd4-389ad489ac4a None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:17.869863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 32/58] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:17.874944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53f0f1-21f1-481f-8b4e-e946ee262f37 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:17.875433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53f0f1-21f1-481f-8b4e-e946ee262f37 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:17.875519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53f0f1-21f1-481f-8b4e-e946ee262f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:17.893777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ac53f0f1-21f1-481f-8b4e-e946ee262f37 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:17.894983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ac53f0f1-21f1-481f-8b4e-e946ee262f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:17.960992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 27/59] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:17.985978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee1a24-7f71-4b7c-bfd4-389ad489ac4a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:18.086091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee1a24-7f71-4b7c-bfd4-389ad489ac4a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:18.110448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:18.119137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce702fa-de29-4656-aa95-2ee96fbcf1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:18.119699 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce702fa-de29-4656-aa95-2ee96fbcf1d1 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:18.119998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce702fa-de29-4656-aa95-2ee96fbcf1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:18.120247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce702fa-de29-4656-aa95-2ee96fbcf1d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:18.121362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce702fa-de29-4656-aa95-2ee96fbcf1d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:18.122829 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce702fa-de29-4656-aa95-2ee96fbcf1d1 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:18.127108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 33/60] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:18.130106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53f0f1-21f1-481f-8b4e-e946ee262f37 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:18.226073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53f0f1-21f1-481f-8b4e-e946ee262f37 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:18.251462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:18.259462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e729a8-6eb2-448b-90e8-d4a788654de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:18.260061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e729a8-6eb2-448b-90e8-d4a788654de5 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:18.260338 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e729a8-6eb2-448b-90e8-d4a788654de5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:18.260606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e729a8-6eb2-448b-90e8-d4a788654de5 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:18.261233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e729a8-6eb2-448b-90e8-d4a788654de5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:18.262715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e729a8-6eb2-448b-90e8-d4a788654de5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:18.379078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 28/61] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:31:18 2022] POST /identity/v3/users => generated 275 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:21.623448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a6ae1-3af3-49e9-9c00-5fe859b1b33f None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:21.624368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a6ae1-3af3-49e9-9c00-5fe859b1b33f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:21.624785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a6ae1-3af3-49e9-9c00-5fe859b1b33f None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:21.626330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 34/62] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:21.632323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3946-6e4f-4016-bb7e-e23444e4ca7a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:21.632792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3946-6e4f-4016-bb7e-e23444e4ca7a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:21.633005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3946-6e4f-4016-bb7e-e23444e4ca7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:21.656926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-60da3946-6e4f-4016-bb7e-e23444e4ca7a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:21.658163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-60da3946-6e4f-4016-bb7e-e23444e4ca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:21.746837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 29/63] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:21.777700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dab716-1075-44db-8866-9b5b00297714 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:21.778354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dab716-1075-44db-8866-9b5b00297714 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:21.778830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dab716-1075-44db-8866-9b5b00297714 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:21.780658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 35/64] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:21.783511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da3946-6e4f-4016-bb7e-e23444e4ca7a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:21.786403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5161-4993-4345-94c6-e7f27f0f4c38 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:21.786933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5161-4993-4345-94c6-e7f27f0f4c38 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:21.787408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5161-4993-4345-94c6-e7f27f0f4c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:21.815753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a06b5161-4993-4345-94c6-e7f27f0f4c38 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:21.817290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a06b5161-4993-4345-94c6-e7f27f0f4c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:21.895471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da3946-6e4f-4016-bb7e-e23444e4ca7a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:21.905662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 36/65] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:21.907751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30290138-0f05-4db6-80c0-cc1705c38989 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:21.908181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30290138-0f05-4db6-80c0-cc1705c38989 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:21.909010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30290138-0f05-4db6-80c0-cc1705c38989 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:21.910742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 37/66] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:21.916848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:21.918661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf6268-8217-45b4-a3b0-0b315d8213de None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:21.919078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf6268-8217-45b4-a3b0-0b315d8213de None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:21.919485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf6268-8217-45b4-a3b0-0b315d8213de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:21.939519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-bcaf6268-8217-45b4-a3b0-0b315d8213de None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:21.941105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bcaf6268-8217-45b4-a3b0-0b315d8213de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:21.953721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f45eda-36b8-49a0-96ef-893798f3fe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:21.954785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f45eda-36b8-49a0-96ef-893798f3fe54 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:21.955274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f45eda-36b8-49a0-96ef-893798f3fe54 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:21.955902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f45eda-36b8-49a0-96ef-893798f3fe54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:21.968922 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f45eda-36b8-49a0-96ef-893798f3fe54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-30f45eda-36b8-49a0-96ef-893798f3fe54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:21.974217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:21.975145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:21.976393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:21.976975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 30/67] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:31:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:21.977639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f45eda-36b8-49a0-96ef-893798f3fe54 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.021614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 38/68] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:22.023589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfaa36-a950-4db3-a008-e6338a8d7cfa None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.024053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfaa36-a950-4db3-a008-e6338a8d7cfa None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.024527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dfaa36-a950-4db3-a008-e6338a8d7cfa None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.025974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 39/69] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:22.026833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfaa36-a950-4db3-a008-e6338a8d7cfa None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.093579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:22.098700 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f45eda-36b8-49a0-96ef-893798f3fe54 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:22.132191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1472eb-27cc-49be-8f68-0075a4f3a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:22.132817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1472eb-27cc-49be-8f68-0075a4f3a915 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.133093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1472eb-27cc-49be-8f68-0075a4f3a915 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.133367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1472eb-27cc-49be-8f68-0075a4f3a915 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.139157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dfaa36-a950-4db3-a008-e6338a8d7cfa None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:22.143459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1472eb-27cc-49be-8f68-0075a4f3a915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-ee1472eb-27cc-49be-8f68-0075a4f3a915 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:22.148840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:22.149743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:22.150606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 31/70] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:31:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:22.151676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742285c-a9cd-4f97-9312-81345bf50b59 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.151960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742285c-a9cd-4f97-9312-81345bf50b59 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.152229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742285c-a9cd-4f97-9312-81345bf50b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.169604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-8742285c-a9cd-4f97-9312-81345bf50b59 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:22.170873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8742285c-a9cd-4f97-9312-81345bf50b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:22.174618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc07e38-a789-453e-b386-e2a2fffaad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:22.175220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc07e38-a789-453e-b386-e2a2fffaad98 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.175498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc07e38-a789-453e-b386-e2a2fffaad98 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.175765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc07e38-a789-453e-b386-e2a2fffaad98 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.176422 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc07e38-a789-453e-b386-e2a2fffaad98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:22.176720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc07e38-a789-453e-b386-e2a2fffaad98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:22.178201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc07e38-a789-453e-b386-e2a2fffaad98 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:22.186346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 40/71] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:22.187124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc07e38-a789-453e-b386-e2a2fffaad98 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.198519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:22.238169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 32/72] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:22.243383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e414f-bec7-4983-974a-3f9b60edcea3 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.243969 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e414f-bec7-4983-974a-3f9b60edcea3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.243969 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8e414f-bec7-4983-974a-3f9b60edcea3 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.245473 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 33/73] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:22.246088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e414f-bec7-4983-974a-3f9b60edcea3 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.294087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc07e38-a789-453e-b386-e2a2fffaad98 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:22.310443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:22.328245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232faa1d-fbfa-42f4-b518-ebb1bf363401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:22.328869 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232faa1d-fbfa-42f4-b518-ebb1bf363401 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.329151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232faa1d-fbfa-42f4-b518-ebb1bf363401 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.329418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232faa1d-fbfa-42f4-b518-ebb1bf363401 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.341006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232faa1d-fbfa-42f4-b518-ebb1bf363401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-232faa1d-fbfa-42f4-b518-ebb1bf363401 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:22.346238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:22.347044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:22.347798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 41/74] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:22.348412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232faa1d-fbfa-42f4-b518-ebb1bf363401 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.349066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8e414f-bec7-4983-974a-3f9b60edcea3 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:22.387291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ab2be-c6ec-4c49-8b5a-e0339b29f72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:22.387923 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ab2be-c6ec-4c49-8b5a-e0339b29f72e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.388198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ab2be-c6ec-4c49-8b5a-e0339b29f72e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.388464 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ab2be-c6ec-4c49-8b5a-e0339b29f72e None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.389126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ab2be-c6ec-4c49-8b5a-e0339b29f72e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:22.389419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ab2be-c6ec-4c49-8b5a-e0339b29f72e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:22.390823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ab2be-c6ec-4c49-8b5a-e0339b29f72e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:22.400170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 34/75] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:22.401318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6b4f2-766d-47f7-be76-4786d76113d7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.401601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6b4f2-766d-47f7-be76-4786d76113d7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.401872 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6b4f2-766d-47f7-be76-4786d76113d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.419217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-deb6b4f2-766d-47f7-be76-4786d76113d7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:22.420408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-deb6b4f2-766d-47f7-be76-4786d76113d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:22.441539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:22.453979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232faa1d-fbfa-42f4-b518-ebb1bf363401 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:22.487127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812f611-d047-4d20-a099-f0e3474a6f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:22.487743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812f611-d047-4d20-a099-f0e3474a6f25 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.488016 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812f611-d047-4d20-a099-f0e3474a6f25 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.488283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812f611-d047-4d20-a099-f0e3474a6f25 None admin] PATH_INFO: `/v3/users/ea8ba450a8204c87830e1b4dce2332ee` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.489382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812f611-d047-4d20-a099-f0e3474a6f25 None admin] RBAC: Authorizing `identity:get_user(user_id=ea8ba450a8204c87830e1b4dce2332ee)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:22.490136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 35/76] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:22.491492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812f611-d047-4d20-a099-f0e3474a6f25 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:22.493105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 42/77] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/users/ea8ba450a8204c87830e1b4dce2332ee => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:22.494232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6b4f2-766d-47f7-be76-4786d76113d7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.495031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812f611-d047-4d20-a099-f0e3474a6f25 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.546359 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:22.603407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812f611-d047-4d20-a099-f0e3474a6f25 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:22.605313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6b4f2-766d-47f7-be76-4786d76113d7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:22.636538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca48b7-03e7-482a-bf1f-1c41a2bb1778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:22.637142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca48b7-03e7-482a-bf1f-1c41a2bb1778 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.637425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca48b7-03e7-482a-bf1f-1c41a2bb1778 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.637696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca48b7-03e7-482a-bf1f-1c41a2bb1778 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.638386 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca48b7-03e7-482a-bf1f-1c41a2bb1778 None admin] RBAC: Adding query filter params (name=member) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:22.638682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca48b7-03e7-482a-bf1f-1c41a2bb1778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:22.640057 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca48b7-03e7-482a-bf1f-1c41a2bb1778 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:22.640250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7c53a-d60a-4965-9dbf-7fd9149cc92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:22.640872 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7c53a-d60a-4965-9dbf-7fd9149cc92d None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.641149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7c53a-d60a-4965-9dbf-7fd9149cc92d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.641417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7c53a-d60a-4965-9dbf-7fd9149cc92d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.648667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 43/78] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:22.649469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca48b7-03e7-482a-bf1f-1c41a2bb1778 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.653752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7c53a-d60a-4965-9dbf-7fd9149cc92d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-76c7c53a-d60a-4965-9dbf-7fd9149cc92d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:22.658571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:22.659471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:22.660398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 36/79] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:22.661196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7c53a-d60a-4965-9dbf-7fd9149cc92d None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.682385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:22.752881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca48b7-03e7-482a-bf1f-1c41a2bb1778 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:22.759988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:22.765164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7c53a-d60a-4965-9dbf-7fd9149cc92d None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:22.786962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2dc2b-6cd6-44e6-9331-1d09a01a61ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:22.788207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2dc2b-6cd6-44e6-9331-1d09a01a61ea None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.788511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2dc2b-6cd6-44e6-9331-1d09a01a61ea None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.788781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2dc2b-6cd6-44e6-9331-1d09a01a61ea None admin] PATH_INFO: `/v3/users/2407f024efda4194a4845b435b0ac7b0` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.800120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a36ce5-188d-45f3-9230-81b4a2cf0852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:22.800719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a36ce5-188d-45f3-9230-81b4a2cf0852 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.800996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a36ce5-188d-45f3-9230-81b4a2cf0852 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.801266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a36ce5-188d-45f3-9230-81b4a2cf0852 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.802205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2dc2b-6cd6-44e6-9331-1d09a01a61ea None admin] RBAC: Authorizing `identity:get_user(user_id=2407f024efda4194a4845b435b0ac7b0)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:22.803858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b2dc2b-6cd6-44e6-9331-1d09a01a61ea None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:22.805508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 44/80] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/users/2407f024efda4194a4845b435b0ac7b0 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:22.806323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2dc2b-6cd6-44e6-9331-1d09a01a61ea None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.869923 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a36ce5-188d-45f3-9230-81b4a2cf0852 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:22.871380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a36ce5-188d-45f3-9230-81b4a2cf0852 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:22.888238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:22.905494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2dc2b-6cd6-44e6-9331-1d09a01a61ea None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:22.939046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c10cdd-843b-4eb2-8858-61669de9dd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:22.939666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c10cdd-843b-4eb2-8858-61669de9dd02 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:22.939947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c10cdd-843b-4eb2-8858-61669de9dd02 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:22.940220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c10cdd-843b-4eb2-8858-61669de9dd02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:22.941329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c10cdd-843b-4eb2-8858-61669de9dd02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:22.942817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c10cdd-843b-4eb2-8858-61669de9dd02 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:22.944911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 45/81] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:22.945674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c10cdd-843b-4eb2-8858-61669de9dd02 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:22.978014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:22.978621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 37/82] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:22.982499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a36ce5-188d-45f3-9230-81b4a2cf0852 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:23.046369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c10cdd-843b-4eb2-8858-61669de9dd02 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:23.079893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a36ce5-188d-45f3-9230-81b4a2cf0852 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:23.080233 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dd672-7de1-4a96-8cb1-892f2b3d3563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:23.080845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dd672-7de1-4a96-8cb1-892f2b3d3563 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:23.081120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dd672-7de1-4a96-8cb1-892f2b3d3563 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:23.081391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dd672-7de1-4a96-8cb1-892f2b3d3563 None admin] PATH_INFO: `/v3/users/2407f024efda4194a4845b435b0ac7b0` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:23.082532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312dd672-7de1-4a96-8cb1-892f2b3d3563 None admin] RBAC: Authorizing `identity:get_user(user_id=2407f024efda4194a4845b435b0ac7b0)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:23.083952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312dd672-7de1-4a96-8cb1-892f2b3d3563 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:23.086222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 46/83] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/users/2407f024efda4194a4845b435b0ac7b0 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:23.086904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dd672-7de1-4a96-8cb1-892f2b3d3563 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:23.101693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:23.119145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a30c9e-8290-4db8-9fc0-5032eb28c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:23.119841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a30c9e-8290-4db8-9fc0-5032eb28c45f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:23.120168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a30c9e-8290-4db8-9fc0-5032eb28c45f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:23.120389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a30c9e-8290-4db8-9fc0-5032eb28c45f None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:23.121055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a30c9e-8290-4db8-9fc0-5032eb28c45f None admin] RBAC: Adding query filter params (name=member) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:23.121356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a30c9e-8290-4db8-9fc0-5032eb28c45f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:23.122801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a30c9e-8290-4db8-9fc0-5032eb28c45f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:23.132462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 38/84] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:23.132462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a30c9e-8290-4db8-9fc0-5032eb28c45f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:23.196201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:23.199112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dd672-7de1-4a96-8cb1-892f2b3d3563 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:23.231946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30552556-c40d-4475-85e4-9ed6f92c479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:23.232541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30552556-c40d-4475-85e4-9ed6f92c479e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:23.232824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30552556-c40d-4475-85e4-9ed6f92c479e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:23.233096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30552556-c40d-4475-85e4-9ed6f92c479e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:23.239792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a30c9e-8290-4db8-9fc0-5032eb28c45f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:23.273203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80021-0b44-4bcd-aa32-f9dda42e8823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:23.273799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80021-0b44-4bcd-aa32-f9dda42e8823 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:23.274103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80021-0b44-4bcd-aa32-f9dda42e8823 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:23.274374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80021-0b44-4bcd-aa32-f9dda42e8823 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:23.275005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff80021-0b44-4bcd-aa32-f9dda42e8823 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:23.276431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff80021-0b44-4bcd-aa32-f9dda42e8823 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:23.300599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30552556-c40d-4475-85e4-9ed6f92c479e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:23.302086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30552556-c40d-4475-85e4-9ed6f92c479e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:23.383420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 39/85] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:31:23 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:23.384440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80021-0b44-4bcd-aa32-f9dda42e8823 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:23.404831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:23.408996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 47/86] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:23.410492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30552556-c40d-4475-85e4-9ed6f92c479e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:23.433545 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:23.515277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80021-0b44-4bcd-aa32-f9dda42e8823 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:23.532658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30552556-c40d-4475-85e4-9ed6f92c479e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:23.564124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c7035-a9b3-4b10-9a7c-c491e6f805df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:23.564769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c7035-a9b3-4b10-9a7c-c491e6f805df None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:23.565150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c7035-a9b3-4b10-9a7c-c491e6f805df None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:23.565319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709c7035-a9b3-4b10-9a7c-c491e6f805df None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:23.566554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c7035-a9b3-4b10-9a7c-c491e6f805df None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:23.568494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709c7035-a9b3-4b10-9a7c-c491e6f805df None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:23.569642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 40/87] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:23 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:23.572559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992344e-e24b-4bfd-b281-90eca3f49394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:23.572717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c7035-a9b3-4b10-9a7c-c491e6f805df None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:23.572804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992344e-e24b-4bfd-b281-90eca3f49394 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:23.572804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992344e-e24b-4bfd-b281-90eca3f49394 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:23.573053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992344e-e24b-4bfd-b281-90eca3f49394 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:23.658595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:23.663002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992344e-e24b-4bfd-b281-90eca3f49394 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:23.664521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992344e-e24b-4bfd-b281-90eca3f49394 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:23.706411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709c7035-a9b3-4b10-9a7c-c491e6f805df None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:23.708773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:23.750509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd0094-32e9-4a10-b04a-af9e028c65f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:23.750753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd0094-32e9-4a10-b04a-af9e028c65f5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:23.751040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd0094-32e9-4a10-b04a-af9e028c65f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:23.751375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd0094-32e9-4a10-b04a-af9e028c65f5 None admin] PATH_INFO: `/v3/users/2407f024efda4194a4845b435b0ac7b0` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:23.752558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd0094-32e9-4a10-b04a-af9e028c65f5 None admin] RBAC: Authorizing `identity:get_user(user_id=2407f024efda4194a4845b435b0ac7b0)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:23.754191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd0094-32e9-4a10-b04a-af9e028c65f5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:23.756560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 41/88] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:23 2022] GET /identity/v3/users/2407f024efda4194a4845b435b0ac7b0 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:23.756999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd0094-32e9-4a10-b04a-af9e028c65f5 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:23.783238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 48/89] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:23.784370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992344e-e24b-4bfd-b281-90eca3f49394 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:23.867090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd0094-32e9-4a10-b04a-af9e028c65f5 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:23.886079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:23.889181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992344e-e24b-4bfd-b281-90eca3f49394 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:23.904142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d92bbd-3c2e-4ac1-b1e5-d052b7e4ff85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:23.904777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d92bbd-3c2e-4ac1-b1e5-d052b7e4ff85 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:23.905062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d92bbd-3c2e-4ac1-b1e5-d052b7e4ff85 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:23.905337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d92bbd-3c2e-4ac1-b1e5-d052b7e4ff85 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:23.906531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d92bbd-3c2e-4ac1-b1e5-d052b7e4ff85 None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:23.908012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d92bbd-3c2e-4ac1-b1e5-d052b7e4ff85 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:23.909919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 42/90] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:23 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:23.916209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d92bbd-3c2e-4ac1-b1e5-d052b7e4ff85 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:23.927659 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fddeee-30ce-4986-8671-f1712bbbbbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:23.929167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fddeee-30ce-4986-8671-f1712bbbbbb0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:23.929585 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fddeee-30ce-4986-8671-f1712bbbbbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:23.930010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fddeee-30ce-4986-8671-f1712bbbbbb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:23.930865 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fddeee-30ce-4986-8671-f1712bbbbbb0 None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=28843c17667d4709ab63a1378bc693a1, user.id=ea8ba450a8204c87830e1b4dce2332ee) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:23.931304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fddeee-30ce-4986-8671-f1712bbbbbb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:23.932880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fddeee-30ce-4986-8671-f1712bbbbbb0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:23.939136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 49/91] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:23 2022] GET /identity/v3/role_assignments?user.id=ea8ba450a8204c87830e1b4dce2332ee&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:23.940189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fddeee-30ce-4986-8671-f1712bbbbbb0 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:23.972963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:24.038095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d92bbd-3c2e-4ac1-b1e5-d052b7e4ff85 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:24.078171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d287f-dcdb-4b03-b995-8e43de09baa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:24.078171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d287f-dcdb-4b03-b995-8e43de09baa6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:24.078171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d287f-dcdb-4b03-b995-8e43de09baa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:24.078171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d287f-dcdb-4b03-b995-8e43de09baa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:24.082591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fddeee-30ce-4986-8671-f1712bbbbbb0 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:24.124091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fac37-c257-4283-acc8-7581de47aaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:24.125018 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fac37-c257-4283-acc8-7581de47aaa6 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:24.125303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fac37-c257-4283-acc8-7581de47aaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:24.125575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fac37-c257-4283-acc8-7581de47aaa6 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:24.126834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fac37-c257-4283-acc8-7581de47aaa6 None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:24.128386 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fac37-c257-4283-acc8-7581de47aaa6 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:24.130192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 50/92] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:23 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:24.131341 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fac37-c257-4283-acc8-7581de47aaa6 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:24.191079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:24.242800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d287f-dcdb-4b03-b995-8e43de09baa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:24.244276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d287f-dcdb-4b03-b995-8e43de09baa6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:24.266646 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fac37-c257-4283-acc8-7581de47aaa6 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:24.335066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495b182-da8b-400a-8170-2704bcccf6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:24.335685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495b182-da8b-400a-8170-2704bcccf6c3 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:24.335971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495b182-da8b-400a-8170-2704bcccf6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:24.336330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495b182-da8b-400a-8170-2704bcccf6c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:24.337074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495b182-da8b-400a-8170-2704bcccf6c3 None admin] RBAC: Adding query filter params (role.id=20d35b10934447938237c0db5202bbfb, scope.project.id=28843c17667d4709ab63a1378bc693a1, user.id=2407f024efda4194a4845b435b0ac7b0) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:24.337382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495b182-da8b-400a-8170-2704bcccf6c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:24.338884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495b182-da8b-400a-8170-2704bcccf6c3 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:24.345339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 51/93] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:24 2022] GET /identity/v3/role_assignments?user.id=2407f024efda4194a4845b435b0ac7b0&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=20d35b10934447938237c0db5202bbfb => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:24.346351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:24.350232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495b182-da8b-400a-8170-2704bcccf6c3 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:24.555970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:24.563469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 43/94] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:24.564823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d287f-dcdb-4b03-b995-8e43de09baa6 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:24.571114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495b182-da8b-400a-8170-2704bcccf6c3 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:24.674726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9060c-7a0f-479b-98e2-dbd1ce84c68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:24.675348 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9060c-7a0f-479b-98e2-dbd1ce84c68b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:24.675608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9060c-7a0f-479b-98e2-dbd1ce84c68b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:24.675879 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9060c-7a0f-479b-98e2-dbd1ce84c68b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:24.676636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9060c-7a0f-479b-98e2-dbd1ce84c68b None admin] RBAC: Adding query filter params (role.id=2490390ea61f4cf594df3587e7674650, scope.project.id=28843c17667d4709ab63a1378bc693a1, user.id=2407f024efda4194a4845b435b0ac7b0) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:24.676922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9060c-7a0f-479b-98e2-dbd1ce84c68b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:24.679067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9060c-7a0f-479b-98e2-dbd1ce84c68b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:24.696137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 52/95] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:24 2022] GET /identity/v3/role_assignments?user.id=2407f024efda4194a4845b435b0ac7b0&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=2490390ea61f4cf594df3587e7674650 => generated 259 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:24.719190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d287f-dcdb-4b03-b995-8e43de09baa6 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:24.760096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52f17c-9122-4dd5-a5d5-48f7563936a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:24.760746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52f17c-9122-4dd5-a5d5-48f7563936a1 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:24.761026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52f17c-9122-4dd5-a5d5-48f7563936a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:24.761302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52f17c-9122-4dd5-a5d5-48f7563936a1 None admin] PATH_INFO: `/v3/projects/724cf809db59488aa5a4aaa19db80a11` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:24.762527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52f17c-9122-4dd5-a5d5-48f7563936a1 None admin] RBAC: Authorizing `identity:get_project(project_id=724cf809db59488aa5a4aaa19db80a11)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:24.763977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52f17c-9122-4dd5-a5d5-48f7563936a1 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:24.766197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 44/96] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:24 2022] GET /identity/v3/projects/724cf809db59488aa5a4aaa19db80a11 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:24.777234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9060c-7a0f-479b-98e2-dbd1ce84c68b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:24.954113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9060c-7a0f-479b-98e2-dbd1ce84c68b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:24.999639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d428a-ac27-496b-bbcc-cbb14bd80359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:25.000251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d428a-ac27-496b-bbcc-cbb14bd80359 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:25.000533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d428a-ac27-496b-bbcc-cbb14bd80359 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:25.000794 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d428a-ac27-496b-bbcc-cbb14bd80359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:25.001524 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d428a-ac27-496b-bbcc-cbb14bd80359 None admin] RBAC: Adding query filter params (role.id=2490390ea61f4cf594df3587e7674650, scope.project.id=724cf809db59488aa5a4aaa19db80a11, user.id=2407f024efda4194a4845b435b0ac7b0) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:25.001827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d428a-ac27-496b-bbcc-cbb14bd80359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:25.003292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66d428a-ac27-496b-bbcc-cbb14bd80359 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:25.014285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 53/97] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:24 2022] GET /identity/v3/role_assignments?user.id=2407f024efda4194a4845b435b0ac7b0&scope.project.id=724cf809db59488aa5a4aaa19db80a11&role.id=2490390ea61f4cf594df3587e7674650 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:26.170720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2009da-bdb1-46ae-bf91-bb7cd0346679 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:26.171747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2009da-bdb1-46ae-bf91-bb7cd0346679 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:26.172228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2009da-bdb1-46ae-bf91-bb7cd0346679 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:26.173767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 45/98] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:26.179783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eb79d-aa18-48fd-8782-ccd2ead1f815 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:26.180640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eb79d-aa18-48fd-8782-ccd2ead1f815 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:26.181140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6eb79d-aa18-48fd-8782-ccd2ead1f815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:26.205589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ff6eb79d-aa18-48fd-8782-ccd2ead1f815 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:26.207157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ff6eb79d-aa18-48fd-8782-ccd2ead1f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:26.231273 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:26.294847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 54/99] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:26.330409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2009da-bdb1-46ae-bf91-bb7cd0346679 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:26.341061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:26.381259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafa3e4-866f-4b0b-ba21-a43bb95ab6c5 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:26.381583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafa3e4-866f-4b0b-ba21-a43bb95ab6c5 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:26.381819 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafa3e4-866f-4b0b-ba21-a43bb95ab6c5 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:26.383204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 55/100] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:26.391761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990e512-c4e2-4fee-8967-be9b09a96013 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:26.392054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990e512-c4e2-4fee-8967-be9b09a96013 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:26.392703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990e512-c4e2-4fee-8967-be9b09a96013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:26.423312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8990e512-c4e2-4fee-8967-be9b09a96013 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:26.424671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8990e512-c4e2-4fee-8967-be9b09a96013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:26.522071 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 56/101] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:26.553621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990e512-c4e2-4fee-8967-be9b09a96013 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:26.562207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2009da-bdb1-46ae-bf91-bb7cd0346679 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:26.638077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc33c82-1896-47fe-81c0-90a3e67d0707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:26.639104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc33c82-1896-47fe-81c0-90a3e67d0707 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:26.639581 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc33c82-1896-47fe-81c0-90a3e67d0707 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:26.640047 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc33c82-1896-47fe-81c0-90a3e67d0707 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:26.662997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc33c82-1896-47fe-81c0-90a3e67d0707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-ddc33c82-1896-47fe-81c0-90a3e67d0707 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:26.670735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:26.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:26.672510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:26.674941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 46/102] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:26.680717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc33c82-1896-47fe-81c0-90a3e67d0707 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:26.682002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:26.705722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990e512-c4e2-4fee-8967-be9b09a96013 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:26.762712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496abf50-1f66-4c59-a77c-12289be118de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:26.763356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496abf50-1f66-4c59-a77c-12289be118de None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:26.763636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496abf50-1f66-4c59-a77c-12289be118de None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:26.763909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496abf50-1f66-4c59-a77c-12289be118de None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:26.765567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496abf50-1f66-4c59-a77c-12289be118de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:26.767227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496abf50-1f66-4c59-a77c-12289be118de None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:26.768970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 57/103] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:26.775903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496abf50-1f66-4c59-a77c-12289be118de None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:26.890354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:26.950899 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc33c82-1896-47fe-81c0-90a3e67d0707 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:26.958372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496abf50-1f66-4c59-a77c-12289be118de None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:27.022875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6807537-4d61-4503-87fa-09f4aaa53ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:27.023608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6807537-4d61-4503-87fa-09f4aaa53ac6 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.023901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6807537-4d61-4503-87fa-09f4aaa53ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.024240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6807537-4d61-4503-87fa-09f4aaa53ac6 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.024918 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6807537-4d61-4503-87fa-09f4aaa53ac6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:27.026678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6807537-4d61-4503-87fa-09f4aaa53ac6 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:27.028574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f5f9d-e1c2-403a-830e-2d123b0cdfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:27.029184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f5f9d-e1c2-403a-830e-2d123b0cdfd3 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.029460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f5f9d-e1c2-403a-830e-2d123b0cdfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.029726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f5f9d-e1c2-403a-830e-2d123b0cdfd3 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.030415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f5f9d-e1c2-403a-830e-2d123b0cdfd3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:27.030712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f5f9d-e1c2-403a-830e-2d123b0cdfd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:27.032066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f5f9d-e1c2-403a-830e-2d123b0cdfd3 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:27.046041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 47/104] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:27.047302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b794759-85be-4b55-858d-8432f285389a None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.047582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b794759-85be-4b55-858d-8432f285389a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.047850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b794759-85be-4b55-858d-8432f285389a None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.049039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 48/105] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:27.055168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b794759-85be-4b55-858d-8432f285389a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:27.139710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 58/106] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:31:26 2022] POST /identity/v3/projects => generated 302 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:31:27.141123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fef0c-d802-4b2a-8685-1ab34ca87064 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.141418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fef0c-d802-4b2a-8685-1ab34ca87064 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.141695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fef0c-d802-4b2a-8685-1ab34ca87064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.163255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:27.188666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-0c2fef0c-d802-4b2a-8685-1ab34ca87064 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:27.189964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0c2fef0c-d802-4b2a-8685-1ab34ca87064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:27.190724 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b794759-85be-4b55-858d-8432f285389a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:27.230346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:27.246452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58741308-c709-4ba0-834d-ee244f5314e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:27.247171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58741308-c709-4ba0-834d-ee244f5314e5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.247456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58741308-c709-4ba0-834d-ee244f5314e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.247726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58741308-c709-4ba0-834d-ee244f5314e5 None admin] PATH_INFO: `/v3/users/0f2132533f50466da1e02d5503f6d57c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.278949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58741308-c709-4ba0-834d-ee244f5314e5 None admin] RBAC: Authorizing `identity:get_user(user_id=0f2132533f50466da1e02d5503f6d57c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:27.280642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58741308-c709-4ba0-834d-ee244f5314e5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:27.282977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 49/107] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:27 2022] GET /identity/v3/users/0f2132533f50466da1e02d5503f6d57c => generated 289 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:27.289457 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58741308-c709-4ba0-834d-ee244f5314e5 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:27.366126 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 59/108] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:27.391066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fef0c-d802-4b2a-8685-1ab34ca87064 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:27.473045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58741308-c709-4ba0-834d-ee244f5314e5 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:27.516853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb2cbc-8c15-4286-9aeb-ff6622bd9e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:27.517426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb2cbc-8c15-4286-9aeb-ff6622bd9e23 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.517659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb2cbc-8c15-4286-9aeb-ff6622bd9e23 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.518065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb2cbc-8c15-4286-9aeb-ff6622bd9e23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.545325 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fef0c-d802-4b2a-8685-1ab34ca87064 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:27.579861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:27.586465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab299b4-a09a-4d53-a61c-922b2cc1e7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:27.587761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab299b4-a09a-4d53-a61c-922b2cc1e7d8 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.588684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab299b4-a09a-4d53-a61c-922b2cc1e7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.589129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab299b4-a09a-4d53-a61c-922b2cc1e7d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.591888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:27.597753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb2cbc-8c15-4286-9aeb-ff6622bd9e23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:27.601060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb2cbc-8c15-4286-9aeb-ff6622bd9e23 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:27.605319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab299b4-a09a-4d53-a61c-922b2cc1e7d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-fab299b4-a09a-4d53-a61c-922b2cc1e7d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:27.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:27.612431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:27.613176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:27.616949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 60/109] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:31:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:27.619496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2380daa-2383-406d-bfa6-5626cc583bb7 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.619974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2380daa-2383-406d-bfa6-5626cc583bb7 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.620389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2380daa-2383-406d-bfa6-5626cc583bb7 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.622357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 61/110] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:27.623405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2380daa-2383-406d-bfa6-5626cc583bb7 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:27.754514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2380daa-2383-406d-bfa6-5626cc583bb7 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:27.788039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 50/111] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 16:31:27.789884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5285c2-e377-4630-874a-717cbcaddf12 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.790735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5285c2-e377-4630-874a-717cbcaddf12 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.791116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5285c2-e377-4630-874a-717cbcaddf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.803050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887cbad-78db-45b9-81a4-1168b3c57a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:27.803663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887cbad-78db-45b9-81a4-1168b3c57a44 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.803948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887cbad-78db-45b9-81a4-1168b3c57a44 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.804221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887cbad-78db-45b9-81a4-1168b3c57a44 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.804901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887cbad-78db-45b9-81a4-1168b3c57a44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:27.805206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887cbad-78db-45b9-81a4-1168b3c57a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:27.806700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887cbad-78db-45b9-81a4-1168b3c57a44 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:27.815079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9b5285c2-e377-4630-874a-717cbcaddf12 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:27.816485 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9b5285c2-e377-4630-874a-717cbcaddf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:27.816860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 62/112] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:27.824235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0071953-e6f4-4074-b880-1dbb82b369b2 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:27.824537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0071953-e6f4-4074-b880-1dbb82b369b2 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:27.824821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0071953-e6f4-4074-b880-1dbb82b369b2 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:27.826192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 63/113] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:27.826922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0071953-e6f4-4074-b880-1dbb82b369b2 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:27.908765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:27.914801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 51/114] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:27.915777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5285c2-e377-4630-874a-717cbcaddf12 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:27.949522 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:27.970816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0071953-e6f4-4074-b880-1dbb82b369b2 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:28.005342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e096c-7486-4c70-8428-e9cadd1e0480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:28.006015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e096c-7486-4c70-8428-e9cadd1e0480 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.006301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e096c-7486-4c70-8428-e9cadd1e0480 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.006576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e096c-7486-4c70-8428-e9cadd1e0480 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.007708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e096c-7486-4c70-8428-e9cadd1e0480 None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:28.009200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e096c-7486-4c70-8428-e9cadd1e0480 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:28.014021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 64/115] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:27 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:28.014021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952f323-5524-4e7d-9d50-f39077f76d40 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.014021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952f323-5524-4e7d-9d50-f39077f76d40 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.014021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952f323-5524-4e7d-9d50-f39077f76d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.032195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-3952f323-5524-4e7d-9d50-f39077f76d40 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:28.033527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3952f323-5524-4e7d-9d50-f39077f76d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:28.039774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5285c2-e377-4630-874a-717cbcaddf12 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:28.076809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e44e4-020d-4267-b680-5d5e6be19bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:28.077447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e44e4-020d-4267-b680-5d5e6be19bb2 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.077726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e44e4-020d-4267-b680-5d5e6be19bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.078039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e44e4-020d-4267-b680-5d5e6be19bb2 None admin] PATH_INFO: `/v3/users/ea8ba450a8204c87830e1b4dce2332ee` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.079185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e44e4-020d-4267-b680-5d5e6be19bb2 None admin] RBAC: Authorizing `identity:get_user(user_id=ea8ba450a8204c87830e1b4dce2332ee)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:28.080637 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e44e4-020d-4267-b680-5d5e6be19bb2 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:28.082404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 52/116] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:27 2022] GET /identity/v3/users/ea8ba450a8204c87830e1b4dce2332ee => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:28.083008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e44e4-020d-4267-b680-5d5e6be19bb2 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:28.110309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 65/117] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:28.111391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1abce-d8de-4b83-b81b-a6dc0aa2a7a0 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.111677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1abce-d8de-4b83-b81b-a6dc0aa2a7a0 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.111949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1abce-d8de-4b83-b81b-a6dc0aa2a7a0 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.117480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 66/118] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:28.117480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1abce-d8de-4b83-b81b-a6dc0aa2a7a0 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:28.179154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:28.196574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:28.215084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e44e4-020d-4267-b680-5d5e6be19bb2 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:28.227050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1abce-d8de-4b83-b81b-a6dc0aa2a7a0 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:28.251184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51b50a-e38b-4004-82c9-04ce32d6e693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:28.252111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51b50a-e38b-4004-82c9-04ce32d6e693 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.252655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51b50a-e38b-4004-82c9-04ce32d6e693 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.253561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51b50a-e38b-4004-82c9-04ce32d6e693 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.263024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d14a1d-87c8-4fd4-81f8-4b81786754e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:28.263693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d14a1d-87c8-4fd4-81f8-4b81786754e9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.263980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d14a1d-87c8-4fd4-81f8-4b81786754e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.264257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d14a1d-87c8-4fd4-81f8-4b81786754e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.264872 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51b50a-e38b-4004-82c9-04ce32d6e693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:28.264973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d14a1d-87c8-4fd4-81f8-4b81786754e9 None admin] RBAC: Adding query filter params (role.id=d7c03cc554a74d0aad88976354649d2d, scope.project.id=28843c17667d4709ab63a1378bc693a1, user.id=0f2132533f50466da1e02d5503f6d57c) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:28.265279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d14a1d-87c8-4fd4-81f8-4b81786754e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:28.266737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d14a1d-87c8-4fd4-81f8-4b81786754e9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-db51b50a-e38b-4004-82c9-04ce32d6e693 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:28.269599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:28.270587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:28.271929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 53/119] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:28.272555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51b50a-e38b-4004-82c9-04ce32d6e693 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:28.278704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 67/120] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:28 2022] GET /identity/v3/role_assignments?user.id=0f2132533f50466da1e02d5503f6d57c&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=d7c03cc554a74d0aad88976354649d2d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:28.280032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a912f0-34c6-44e2-ba1e-ce9a8bafc36e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.280317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a912f0-34c6-44e2-ba1e-ce9a8bafc36e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.280593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a912f0-34c6-44e2-ba1e-ce9a8bafc36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.299679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-03a912f0-34c6-44e2-ba1e-ce9a8bafc36e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:28.300845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-03a912f0-34c6-44e2-ba1e-ce9a8bafc36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:28.385978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 68/121] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:28.385978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a912f0-34c6-44e2-ba1e-ce9a8bafc36e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:28.447291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51b50a-e38b-4004-82c9-04ce32d6e693 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:28.463505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:28.489046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e24a36-5901-4977-8520-fbb0c69af38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:28.489680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e24a36-5901-4977-8520-fbb0c69af38a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.490505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e24a36-5901-4977-8520-fbb0c69af38a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.491042 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e24a36-5901-4977-8520-fbb0c69af38a None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.492838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e24a36-5901-4977-8520-fbb0c69af38a None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:28.494848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e24a36-5901-4977-8520-fbb0c69af38a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:28.498281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 54/122] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:28 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:28.499582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e24a36-5901-4977-8520-fbb0c69af38a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:28.521839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a912f0-34c6-44e2-ba1e-ce9a8bafc36e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:28.524095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:28.565632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d9aa8-062f-45c6-8348-da457b96f106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:28.566623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d9aa8-062f-45c6-8348-da457b96f106 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.567169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d9aa8-062f-45c6-8348-da457b96f106 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.567647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d9aa8-062f-45c6-8348-da457b96f106 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.582230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d9aa8-062f-45c6-8348-da457b96f106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-289d9aa8-062f-45c6-8348-da457b96f106 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:28.587751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:28.588608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:28.589605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:28.591866 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 69/123] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:31:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:28.592848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d9aa8-062f-45c6-8348-da457b96f106 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:28.643893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e24a36-5901-4977-8520-fbb0c69af38a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:28.718694 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0e56d-d808-4226-9d27-8893ea54f4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:28.719681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0e56d-d808-4226-9d27-8893ea54f4ac None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.720160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0e56d-d808-4226-9d27-8893ea54f4ac None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.720709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0e56d-d808-4226-9d27-8893ea54f4ac None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.721571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0e56d-d808-4226-9d27-8893ea54f4ac None admin] RBAC: Adding query filter params (name=member) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:28.722153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0e56d-d808-4226-9d27-8893ea54f4ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:28.723710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0e56d-d808-4226-9d27-8893ea54f4ac None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:28.733043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 55/124] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:28.738541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0e56d-d808-4226-9d27-8893ea54f4ac None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:28.753812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d9aa8-062f-45c6-8348-da457b96f106 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:28.764243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:28.790845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdb794-709e-4085-af52-958ef153f25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:28.791493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdb794-709e-4085-af52-958ef153f25c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.791769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdb794-709e-4085-af52-958ef153f25c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.792037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdb794-709e-4085-af52-958ef153f25c None admin] PATH_INFO: `/v3/roles/member` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.804134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fdb794-709e-4085-af52-958ef153f25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-a3fdb794-709e-4085-af52-958ef153f25c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:28.808807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:28.809662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:28.810484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 70/125] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:28.811134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdb794-709e-4085-af52-958ef153f25c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:28.826048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:28.858835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0e56d-d808-4226-9d27-8893ea54f4ac None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:28.895308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b84db-1cb9-4931-b81c-3344c1a833c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:28.896426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b84db-1cb9-4931-b81c-3344c1a833c8 None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.896905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b84db-1cb9-4931-b81c-3344c1a833c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.897380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b84db-1cb9-4931-b81c-3344c1a833c8 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1/users/ea8ba450a8204c87830e1b4dce2332ee/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.899569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b84db-1cb9-4931-b81c-3344c1a833c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=28843c17667d4709ab63a1378bc693a1, user_id=ea8ba450a8204c87830e1b4dce2332ee, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:28.901275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b84db-1cb9-4931-b81c-3344c1a833c8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:28.923451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fdb794-709e-4085-af52-958ef153f25c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:28.959310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79469b63-8548-44e6-9a0d-72b8dcbc597e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:28.960300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79469b63-8548-44e6-9a0d-72b8dcbc597e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:28.960802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79469b63-8548-44e6-9a0d-72b8dcbc597e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:28.961374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79469b63-8548-44e6-9a0d-72b8dcbc597e None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:28.962278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79469b63-8548-44e6-9a0d-72b8dcbc597e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:28.962782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79469b63-8548-44e6-9a0d-72b8dcbc597e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:28.964709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79469b63-8548-44e6-9a0d-72b8dcbc597e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:28.974055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 71/126] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:31:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:28.975668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79469b63-8548-44e6-9a0d-72b8dcbc597e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:28.999909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 56/127] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:31:28 2022] PUT /identity/v3/projects/28843c17667d4709ab63a1378bc693a1/users/ea8ba450a8204c87830e1b4dce2332ee/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:31:29.001252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b84db-1cb9-4931-b81c-3344c1a833c8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:29.021833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:29.179024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79469b63-8548-44e6-9a0d-72b8dcbc597e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:29.205393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b84db-1cb9-4931-b81c-3344c1a833c8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:29.238961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fa9e7-7d6b-441b-867b-841a8c3a7330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:29.239592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fa9e7-7d6b-441b-867b-841a8c3a7330 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:29.239906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fa9e7-7d6b-441b-867b-841a8c3a7330 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:29.240240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fa9e7-7d6b-441b-867b-841a8c3a7330 None admin] PATH_INFO: `/v3/users/2407f024efda4194a4845b435b0ac7b0` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:29.241705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fa9e7-7d6b-441b-867b-841a8c3a7330 None admin] RBAC: Authorizing `identity:get_user(user_id=2407f024efda4194a4845b435b0ac7b0)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:29.243381 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fa9e7-7d6b-441b-867b-841a8c3a7330 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:29.245677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 72/128] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:28 2022] GET /identity/v3/users/2407f024efda4194a4845b435b0ac7b0 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:29.247462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fa9e7-7d6b-441b-867b-841a8c3a7330 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:29.257876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc165a39-25ef-438f-a744-cc1249b6d2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:29.259365 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc165a39-25ef-438f-a744-cc1249b6d2d4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:29.259862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc165a39-25ef-438f-a744-cc1249b6d2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:29.260335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc165a39-25ef-438f-a744-cc1249b6d2d4 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:29.261759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc165a39-25ef-438f-a744-cc1249b6d2d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:29.262742 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc165a39-25ef-438f-a744-cc1249b6d2d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:29.264638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc165a39-25ef-438f-a744-cc1249b6d2d4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:29.267125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:29.278630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 57/129] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:29.280351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc165a39-25ef-438f-a744-cc1249b6d2d4 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:29.400890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:29.407010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fa9e7-7d6b-441b-867b-841a8c3a7330 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:29.440921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc165a39-25ef-438f-a744-cc1249b6d2d4 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:29.475686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42760a-6e73-4d5c-afae-6acaa4f59523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:29.476618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42760a-6e73-4d5c-afae-6acaa4f59523 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:29.477063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42760a-6e73-4d5c-afae-6acaa4f59523 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:29.477500 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42760a-6e73-4d5c-afae-6acaa4f59523 None admin] PATH_INFO: `/v3/users/2407f024efda4194a4845b435b0ac7b0` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:29.479257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42760a-6e73-4d5c-afae-6acaa4f59523 None admin] RBAC: Authorizing `identity:get_user(user_id=2407f024efda4194a4845b435b0ac7b0)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:29.482510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42760a-6e73-4d5c-afae-6acaa4f59523 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:29.484298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 73/130] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:29 2022] GET /identity/v3/users/2407f024efda4194a4845b435b0ac7b0 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:29.485284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42760a-6e73-4d5c-afae-6acaa4f59523 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:29.523755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa91ec-9556-467e-97aa-c5be0fe7106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:29.524761 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa91ec-9556-467e-97aa-c5be0fe7106c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:29.525217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa91ec-9556-467e-97aa-c5be0fe7106c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:29.525689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa91ec-9556-467e-97aa-c5be0fe7106c None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:29.527308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa91ec-9556-467e-97aa-c5be0fe7106c None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:29.529250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa91ec-9556-467e-97aa-c5be0fe7106c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:29.532680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 58/131] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:29 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:29.534935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa91ec-9556-467e-97aa-c5be0fe7106c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:29.629144 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42760a-6e73-4d5c-afae-6acaa4f59523 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:29.646918 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:29.656820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa91ec-9556-467e-97aa-c5be0fe7106c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:29.677176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80412dc-577e-4768-ace7-9b9637380f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:29.678205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80412dc-577e-4768-ace7-9b9637380f00 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:29.678640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80412dc-577e-4768-ace7-9b9637380f00 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:29.679063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80412dc-577e-4768-ace7-9b9637380f00 None admin] PATH_INFO: `/v3/users/2407f024efda4194a4845b435b0ac7b0` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:29.686240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80412dc-577e-4768-ace7-9b9637380f00 None admin] RBAC: Authorizing `identity:get_user(user_id=2407f024efda4194a4845b435b0ac7b0)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:29.686240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80412dc-577e-4768-ace7-9b9637380f00 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:29.686240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 74/132] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:29 2022] GET /identity/v3/users/2407f024efda4194a4845b435b0ac7b0 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:29.686240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80412dc-577e-4768-ace7-9b9637380f00 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:29.702442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7a1f6a-7aad-4dbe-98ba-45c35dc467c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:29.703411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a1f6a-7aad-4dbe-98ba-45c35dc467c7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:29.703975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a1f6a-7aad-4dbe-98ba-45c35dc467c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:29.704431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7a1f6a-7aad-4dbe-98ba-45c35dc467c7 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:29.705893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a1f6a-7aad-4dbe-98ba-45c35dc467c7 None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:29.707606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7a1f6a-7aad-4dbe-98ba-45c35dc467c7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:29.710080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 59/133] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:29 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:29.711800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1257207-f48d-4aff-b782-231f81e9519d None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:29.712273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1257207-f48d-4aff-b782-231f81e9519d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:29.712738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1257207-f48d-4aff-b782-231f81e9519d None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:29.714769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 60/134] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:29.715772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1257207-f48d-4aff-b782-231f81e9519d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:29.737129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:29.815056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80412dc-577e-4768-ace7-9b9637380f00 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:29.835661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1257207-f48d-4aff-b782-231f81e9519d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:29.851352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb9f11-f7c0-4e24-88f5-5bf93621d8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:29.852336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb9f11-f7c0-4e24-88f5-5bf93621d8f7 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:29.852831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb9f11-f7c0-4e24-88f5-5bf93621d8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:29.853306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb9f11-f7c0-4e24-88f5-5bf93621d8f7 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1/users/2407f024efda4194a4845b435b0ac7b0/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:29.855522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb9f11-f7c0-4e24-88f5-5bf93621d8f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=28843c17667d4709ab63a1378bc693a1, user_id=2407f024efda4194a4845b435b0ac7b0, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:29.857624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb9f11-f7c0-4e24-88f5-5bf93621d8f7 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:29.870607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:29.876004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f614ae-5f2d-4445-b206-52d4fafb44eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:29.878352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f614ae-5f2d-4445-b206-52d4fafb44eb None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:29.878352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f614ae-5f2d-4445-b206-52d4fafb44eb None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:29.878352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f614ae-5f2d-4445-b206-52d4fafb44eb None admin] PATH_INFO: `/v3/projects/724cf809db59488aa5a4aaa19db80a11` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:29.878705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f614ae-5f2d-4445-b206-52d4fafb44eb None admin] RBAC: Authorizing `identity:get_project(project_id=724cf809db59488aa5a4aaa19db80a11)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:29.881206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f614ae-5f2d-4445-b206-52d4fafb44eb None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:29.883124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 61/135] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:29 2022] GET /identity/v3/projects/724cf809db59488aa5a4aaa19db80a11 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:29.883671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f614ae-5f2d-4445-b206-52d4fafb44eb None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:29.961533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 75/136] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:31:29 2022] PUT /identity/v3/projects/28843c17667d4709ab63a1378bc693a1/users/2407f024efda4194a4845b435b0ac7b0/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:31:29.964399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689128a-b600-4c3d-abb6-bd7f20817468 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:29.965478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689128a-b600-4c3d-abb6-bd7f20817468 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:29.965990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689128a-b600-4c3d-abb6-bd7f20817468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:29.992629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-7689128a-b600-4c3d-abb6-bd7f20817468 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:29.994594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7689128a-b600-4c3d-abb6-bd7f20817468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:30.031085 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f614ae-5f2d-4445-b206-52d4fafb44eb None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:30.084037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 76/137] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:30.085873 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689128a-b600-4c3d-abb6-bd7f20817468 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:30.106966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adadb40-f3c0-42e0-89b3-feeecb52789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:30.107641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adadb40-f3c0-42e0-89b3-feeecb52789a None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:30.108112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adadb40-f3c0-42e0-89b3-feeecb52789a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:30.108385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adadb40-f3c0-42e0-89b3-feeecb52789a None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1/users/2407f024efda4194a4845b435b0ac7b0/roles/20d35b10934447938237c0db5202bbfb` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:30.117802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:30.118092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adadb40-f3c0-42e0-89b3-feeecb52789a None admin] RBAC: Authorizing `identity:create_grant(project_id=28843c17667d4709ab63a1378bc693a1, user_id=2407f024efda4194a4845b435b0ac7b0, role_id=20d35b10934447938237c0db5202bbfb)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:30.120470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adadb40-f3c0-42e0-89b3-feeecb52789a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:30.199173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 62/138] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:31:29 2022] PUT /identity/v3/projects/28843c17667d4709ab63a1378bc693a1/users/2407f024efda4194a4845b435b0ac7b0/roles/20d35b10934447938237c0db5202bbfb => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:31:30.200329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adadb40-f3c0-42e0-89b3-feeecb52789a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:30.210530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689128a-b600-4c3d-abb6-bd7f20817468 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:30.263990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd1326-ea67-46a9-8568-4b4b7ba28fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:30.264843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1326-ea67-46a9-8568-4b4b7ba28fc2 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:30.265220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1326-ea67-46a9-8568-4b4b7ba28fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:30.266040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1326-ea67-46a9-8568-4b4b7ba28fc2 None admin] PATH_INFO: `/v3/projects/724cf809db59488aa5a4aaa19db80a11/users/2407f024efda4194a4845b435b0ac7b0/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:30.268925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd1326-ea67-46a9-8568-4b4b7ba28fc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=724cf809db59488aa5a4aaa19db80a11, user_id=2407f024efda4194a4845b435b0ac7b0, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:30.270976 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd1326-ea67-46a9-8568-4b4b7ba28fc2 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:30.302109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:30.354351 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 77/139] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:31:30 2022] PUT /identity/v3/projects/724cf809db59488aa5a4aaa19db80a11/users/2407f024efda4194a4845b435b0ac7b0/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:31:30.376966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adadb40-f3c0-42e0-89b3-feeecb52789a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:30.422062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc00e97-ef52-4062-9e25-0bc02faa8653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:30.422702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc00e97-ef52-4062-9e25-0bc02faa8653 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:30.423147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc00e97-ef52-4062-9e25-0bc02faa8653 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:30.423429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc00e97-ef52-4062-9e25-0bc02faa8653 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:30.425857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc00e97-ef52-4062-9e25-0bc02faa8653 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:30.427432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc00e97-ef52-4062-9e25-0bc02faa8653 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:30.429025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 63/140] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:30.435592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd1326-ea67-46a9-8568-4b4b7ba28fc2 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:30.534362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:30.544631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed2ca8-a7fd-4693-abf8-0bc03c062624 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:30.544968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed2ca8-a7fd-4693-abf8-0bc03c062624 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:30.545179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed2ca8-a7fd-4693-abf8-0bc03c062624 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:30.546706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 64/141] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:31:30.552635 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e173bb-6361-406d-b936-b4f0b692ea13 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:30.553029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e173bb-6361-406d-b936-b4f0b692ea13 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:30.553268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e173bb-6361-406d-b936-b4f0b692ea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:30.562161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd1326-ea67-46a9-8568-4b4b7ba28fc2 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:30.600427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f5e173bb-6361-406d-b936-b4f0b692ea13 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:30.601792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f5e173bb-6361-406d-b936-b4f0b692ea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:30.669322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01544d-32a8-44e4-b71b-3c025e8a8768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:30.669930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01544d-32a8-44e4-b71b-3c025e8a8768 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:30.670293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01544d-32a8-44e4-b71b-3c025e8a8768 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:30.670557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01544d-32a8-44e4-b71b-3c025e8a8768 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:30.671201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01544d-32a8-44e4-b71b-3c025e8a8768 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:30.672648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01544d-32a8-44e4-b71b-3c025e8a8768 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:30.726022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 65/142] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 09 16:31:30.749476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e173bb-6361-406d-b936-b4f0b692ea13 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:30.796440 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 78/143] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:31:30 2022] POST /identity/v3/users => generated 283 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:30.812209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:30.856221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e173bb-6361-406d-b936-b4f0b692ea13 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:30.895627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741817d-31eb-4396-b2dc-17182b74c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:30.896294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741817d-31eb-4396-b2dc-17182b74c5da None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:30.896589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741817d-31eb-4396-b2dc-17182b74c5da None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:30.896917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741817d-31eb-4396-b2dc-17182b74c5da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:30.909080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741817d-31eb-4396-b2dc-17182b74c5da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-a741817d-31eb-4396-b2dc-17182b74c5da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:30.914536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:30.915418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:30.916214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 66/144] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:31:30.923290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01544d-32a8-44e4-b71b-3c025e8a8768 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:31.065662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01544d-32a8-44e4-b71b-3c025e8a8768 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:31.115306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e52511-c3b9-4060-800e-7e3a0410f71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:31.115921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e52511-c3b9-4060-800e-7e3a0410f71b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:31.116189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e52511-c3b9-4060-800e-7e3a0410f71b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:31.116452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e52511-c3b9-4060-800e-7e3a0410f71b None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:31.117122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e52511-c3b9-4060-800e-7e3a0410f71b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:31.117416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e52511-c3b9-4060-800e-7e3a0410f71b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:31.118841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e52511-c3b9-4060-800e-7e3a0410f71b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:31.131308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 79/145] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:31.136846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741817d-31eb-4396-b2dc-17182b74c5da None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:31.287531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741817d-31eb-4396-b2dc-17182b74c5da None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:31.311800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:31.334269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e1fb1-bd0e-40a6-b0b3-cf4969db2cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:31.334269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e1fb1-bd0e-40a6-b0b3-cf4969db2cb5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:31.334269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e1fb1-bd0e-40a6-b0b3-cf4969db2cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:31.334269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e1fb1-bd0e-40a6-b0b3-cf4969db2cb5 None admin] PATH_INFO: `/v3/users/0f2132533f50466da1e02d5503f6d57c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:31.334653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e1fb1-bd0e-40a6-b0b3-cf4969db2cb5 None admin] RBAC: Authorizing `identity:get_user(user_id=0f2132533f50466da1e02d5503f6d57c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:31.334653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e1fb1-bd0e-40a6-b0b3-cf4969db2cb5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:31.336147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 67/146] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:31 2022] GET /identity/v3/users/0f2132533f50466da1e02d5503f6d57c => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:31.341822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e52511-c3b9-4060-800e-7e3a0410f71b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:31.348529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:31.511667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e52511-c3b9-4060-800e-7e3a0410f71b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:31.548415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed9fdd-7eee-4f31-9429-41d5077c401f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:31.549050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed9fdd-7eee-4f31-9429-41d5077c401f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:31.549323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed9fdd-7eee-4f31-9429-41d5077c401f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:31.549587 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed9fdd-7eee-4f31-9429-41d5077c401f None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:31.550780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed9fdd-7eee-4f31-9429-41d5077c401f None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:31.552862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed9fdd-7eee-4f31-9429-41d5077c401f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:31.554695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 80/147] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:31 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:31.560274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e1fb1-bd0e-40a6-b0b3-cf4969db2cb5 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:31.673380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e1fb1-bd0e-40a6-b0b3-cf4969db2cb5 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:31.710950 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d59360-d360-4ed5-9d01-48395d1f1682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:31.711606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d59360-d360-4ed5-9d01-48395d1f1682 None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:31.711883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d59360-d360-4ed5-9d01-48395d1f1682 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:31.712148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d59360-d360-4ed5-9d01-48395d1f1682 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1/users/0f2132533f50466da1e02d5503f6d57c/roles/d7c03cc554a74d0aad88976354649d2d` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:31.715170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d59360-d360-4ed5-9d01-48395d1f1682 None admin] RBAC: Authorizing `identity:create_grant(project_id=28843c17667d4709ab63a1378bc693a1, user_id=0f2132533f50466da1e02d5503f6d57c, role_id=d7c03cc554a74d0aad88976354649d2d)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:31.716785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d59360-d360-4ed5-9d01-48395d1f1682 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:31.817996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 68/148] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:31:31 2022] PUT /identity/v3/projects/28843c17667d4709ab63a1378bc693a1/users/0f2132533f50466da1e02d5503f6d57c/roles/d7c03cc554a74d0aad88976354649d2d => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:31:31.893137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce09b3-fa00-406e-a889-4e7da2d0cc5f None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:31.893477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce09b3-fa00-406e-a889-4e7da2d0cc5f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:31.893689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ce09b3-fa00-406e-a889-4e7da2d0cc5f None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:31.895355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 81/149] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:31.901148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40671416-88f6-484d-9ba1-cffc1a8bf9e4 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:31.901769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40671416-88f6-484d-9ba1-cffc1a8bf9e4 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:31.902460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40671416-88f6-484d-9ba1-cffc1a8bf9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:31.925224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-40671416-88f6-484d-9ba1-cffc1a8bf9e4 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:31.926879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-40671416-88f6-484d-9ba1-cffc1a8bf9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:31.979663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:32.017331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 69/150] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:32.055389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce09b3-fa00-406e-a889-4e7da2d0cc5f None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:32.158197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:32.207956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ce09b3-fa00-406e-a889-4e7da2d0cc5f None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:32.250297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91398c15-fbb2-44f2-9cec-96880f916fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:32.250938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398c15-fbb2-44f2-9cec-96880f916fbe None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:32.251221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398c15-fbb2-44f2-9cec-96880f916fbe None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:32.251492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91398c15-fbb2-44f2-9cec-96880f916fbe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:32.267841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91398c15-fbb2-44f2-9cec-96880f916fbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-91398c15-fbb2-44f2-9cec-96880f916fbe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:32.273644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:32.274520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:32.275266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 82/151] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:31:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:32.287781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40671416-88f6-484d-9ba1-cffc1a8bf9e4 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:32.409458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40671416-88f6-484d-9ba1-cffc1a8bf9e4 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:32.447908 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacb08a-accf-4eb2-8693-620b14b8a630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:32.448524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb08a-accf-4eb2-8693-620b14b8a630 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:32.448813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb08a-accf-4eb2-8693-620b14b8a630 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:32.449148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faacb08a-accf-4eb2-8693-620b14b8a630 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:32.449809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacb08a-accf-4eb2-8693-620b14b8a630 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:32.450249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacb08a-accf-4eb2-8693-620b14b8a630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:32.451708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faacb08a-accf-4eb2-8693-620b14b8a630 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:32.463281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 70/152] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:32.469213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91398c15-fbb2-44f2-9cec-96880f916fbe None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:32.586727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91398c15-fbb2-44f2-9cec-96880f916fbe None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:32.626021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:32.632524 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9658-8cf8-481b-9a4b-0c2fd60f9987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:32.633166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9658-8cf8-481b-9a4b-0c2fd60f9987 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:32.633472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9658-8cf8-481b-9a4b-0c2fd60f9987 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:32.633737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba9658-8cf8-481b-9a4b-0c2fd60f9987 None admin] PATH_INFO: `/v3/users/ea8ba450a8204c87830e1b4dce2332ee` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:32.634947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9658-8cf8-481b-9a4b-0c2fd60f9987 None admin] RBAC: Authorizing `identity:get_user(user_id=ea8ba450a8204c87830e1b4dce2332ee)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:32.636385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba9658-8cf8-481b-9a4b-0c2fd60f9987 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:32.637978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 83/153] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:32 2022] GET /identity/v3/users/ea8ba450a8204c87830e1b4dce2332ee => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:32.643733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacb08a-accf-4eb2-8693-620b14b8a630 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:32.688203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:32.753681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faacb08a-accf-4eb2-8693-620b14b8a630 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:32.790155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d98fb-a437-4bdc-8161-5cdf768cad80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:32.790767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d98fb-a437-4bdc-8161-5cdf768cad80 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:32.791039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d98fb-a437-4bdc-8161-5cdf768cad80 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:32.791303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d98fb-a437-4bdc-8161-5cdf768cad80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:32.878685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d98fb-a437-4bdc-8161-5cdf768cad80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:32.880649 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d98fb-a437-4bdc-8161-5cdf768cad80 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:32.922266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:32.989340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 71/154] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:32.995675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9658-8cf8-481b-9a4b-0c2fd60f9987 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:33.118043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba9658-8cf8-481b-9a4b-0c2fd60f9987 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:33.157855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f610232-93d6-43bf-b5cf-7867e95a7a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:33.158556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f610232-93d6-43bf-b5cf-7867e95a7a4a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:33.158828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f610232-93d6-43bf-b5cf-7867e95a7a4a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:33.159090 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f610232-93d6-43bf-b5cf-7867e95a7a4a None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:33.160245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f610232-93d6-43bf-b5cf-7867e95a7a4a None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:33.161719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f610232-93d6-43bf-b5cf-7867e95a7a4a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:33.163359 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 84/155] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:32 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:33.169440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d98fb-a437-4bdc-8161-5cdf768cad80 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:33.284259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d98fb-a437-4bdc-8161-5cdf768cad80 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:33.325946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d88747-c172-4139-892f-9ce43009bfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:33.326564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d88747-c172-4139-892f-9ce43009bfe0 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:33.326836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d88747-c172-4139-892f-9ce43009bfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:33.327096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d88747-c172-4139-892f-9ce43009bfe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:33.327816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d88747-c172-4139-892f-9ce43009bfe0 None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=28843c17667d4709ab63a1378bc693a1, user.id=ea8ba450a8204c87830e1b4dce2332ee) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:33.328114 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d88747-c172-4139-892f-9ce43009bfe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:33.329585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d88747-c172-4139-892f-9ce43009bfe0 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:33.335852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 72/156] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:33 2022] GET /identity/v3/role_assignments?user.id=ea8ba450a8204c87830e1b4dce2332ee&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:33.450972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a3694-7e97-4936-83cb-5f2d2ec4242a None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:33.451265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a3694-7e97-4936-83cb-5f2d2ec4242a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:33.451455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a3694-7e97-4936-83cb-5f2d2ec4242a None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:33.453066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 85/157] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:33.462406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d653b-5a11-49ce-bf5e-c9116f7151fb None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:33.462919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d653b-5a11-49ce-bf5e-c9116f7151fb None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:33.462919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d653b-5a11-49ce-bf5e-c9116f7151fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:33.470052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:33.483674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6e3d653b-5a11-49ce-bf5e-c9116f7151fb None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:33.484861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6e3d653b-5a11-49ce-bf5e-c9116f7151fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:33.569410 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 73/158] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:33.610037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a3694-7e97-4936-83cb-5f2d2ec4242a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:33.612710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dbca7-fa97-41a0-b494-3f8a66b009a3 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:33.612992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dbca7-fa97-41a0-b494-3f8a66b009a3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:33.613246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dbca7-fa97-41a0-b494-3f8a66b009a3 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:33.615006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 74/159] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:33.623885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25eccf-e089-46f3-859b-54fa440d5184 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:33.624308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25eccf-e089-46f3-859b-54fa440d5184 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:33.624635 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25eccf-e089-46f3-859b-54fa440d5184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:33.650548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-4d25eccf-e089-46f3-859b-54fa440d5184 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:33.651723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4d25eccf-e089-46f3-859b-54fa440d5184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:33.727270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:33.760641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a3694-7e97-4936-83cb-5f2d2ec4242a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:33.793107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 75/160] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:33.807679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c0f29-dcc5-4940-807f-3f6f2709682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:33.808413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c0f29-dcc5-4940-807f-3f6f2709682c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:33.808591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c0f29-dcc5-4940-807f-3f6f2709682c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:33.808861 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c0f29-dcc5-4940-807f-3f6f2709682c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:33.825767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25eccf-e089-46f3-859b-54fa440d5184 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:33.828568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c0f29-dcc5-4940-807f-3f6f2709682c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-111c0f29-dcc5-4940-807f-3f6f2709682c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:33.833910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:33.834891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:33.835751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 86/161] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:31:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:33.836943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fed77-c149-4e1c-816d-f76adc0ca797 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:33.836943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fed77-c149-4e1c-816d-f76adc0ca797 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:33.837092 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fed77-c149-4e1c-816d-f76adc0ca797 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:33.842585 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 87/162] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:33.843411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fed77-c149-4e1c-816d-f76adc0ca797 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:33.974183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:34.065495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25eccf-e089-46f3-859b-54fa440d5184 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:34.101244 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fed77-c149-4e1c-816d-f76adc0ca797 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:34.147118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:34.153162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d7f64-9df6-4f78-9119-486357c3b7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:34.153803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d7f64-9df6-4f78-9119-486357c3b7d8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:34.154103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d7f64-9df6-4f78-9119-486357c3b7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:34.154373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d7f64-9df6-4f78-9119-486357c3b7d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:34.157187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1758a-a558-4a86-bd87-b1c3fa3a098e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:34.157794 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1758a-a558-4a86-bd87-b1c3fa3a098e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:34.158124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1758a-a558-4a86-bd87-b1c3fa3a098e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:34.158394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1758a-a558-4a86-bd87-b1c3fa3a098e None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:34.159064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1758a-a558-4a86-bd87-b1c3fa3a098e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:34.159364 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1758a-a558-4a86-bd87-b1c3fa3a098e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:34.160738 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1758a-a558-4a86-bd87-b1c3fa3a098e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:34.175699 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 88/163] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:31:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:34.177007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ed55-0e04-453f-ad7a-deec3364d2e1 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:34.177938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ed55-0e04-453f-ad7a-deec3364d2e1 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:34.178238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ed55-0e04-453f-ad7a-deec3364d2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:34.180646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d7f64-9df6-4f78-9119-486357c3b7d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-d19d7f64-9df6-4f78-9119-486357c3b7d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:34.188587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:34.189459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:34.190324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 76/164] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:34.194953 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d7f64-9df6-4f78-9119-486357c3b7d8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:34.211352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-04d5ed55-0e04-453f-ad7a-deec3364d2e1 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:34.212577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-04d5ed55-0e04-453f-ad7a-deec3364d2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:34.322870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 89/165] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:34.323992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ed55-0e04-453f-ad7a-deec3364d2e1 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:34.475163 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ed55-0e04-453f-ad7a-deec3364d2e1 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:34.484957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d7f64-9df6-4f78-9119-486357c3b7d8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:34.520234 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba678d-f274-4b23-bc30-d6fe34a2f5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:34.521430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba678d-f274-4b23-bc30-d6fe34a2f5ad None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:34.521708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba678d-f274-4b23-bc30-d6fe34a2f5ad None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:34.521994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba678d-f274-4b23-bc30-d6fe34a2f5ad None admin] PATH_INFO: `/v3/users/2407f024efda4194a4845b435b0ac7b0` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:34.522381 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:34.526028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba678d-f274-4b23-bc30-d6fe34a2f5ad None admin] RBAC: Authorizing `identity:get_user(user_id=2407f024efda4194a4845b435b0ac7b0)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:34.526028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba678d-f274-4b23-bc30-d6fe34a2f5ad None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:34.526853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 77/166] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:34 2022] GET /identity/v3/users/2407f024efda4194a4845b435b0ac7b0 => generated 275 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:34.527685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba678d-f274-4b23-bc30-d6fe34a2f5ad None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:34.543569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2eabd-dc40-4d8b-a4b8-37753c6d2169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:34.544186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2eabd-dc40-4d8b-a4b8-37753c6d2169 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:34.544465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2eabd-dc40-4d8b-a4b8-37753c6d2169 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:34.544736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea2eabd-dc40-4d8b-a4b8-37753c6d2169 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:34.545410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2eabd-dc40-4d8b-a4b8-37753c6d2169 None admin] RBAC: Adding query filter params (name=member) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:34.545709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2eabd-dc40-4d8b-a4b8-37753c6d2169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:34.547345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea2eabd-dc40-4d8b-a4b8-37753c6d2169 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:34.554183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:34.565883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 90/167] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:34.566615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2eabd-dc40-4d8b-a4b8-37753c6d2169 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:34.651200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba678d-f274-4b23-bc30-d6fe34a2f5ad None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:34.689736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7476c87-ea8a-47ba-8d02-464c38e167f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:34.690404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7476c87-ea8a-47ba-8d02-464c38e167f1 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:34.690748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7476c87-ea8a-47ba-8d02-464c38e167f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:34.691039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7476c87-ea8a-47ba-8d02-464c38e167f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:34.702117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7476c87-ea8a-47ba-8d02-464c38e167f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-f7476c87-ea8a-47ba-8d02-464c38e167f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:34.707185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:34.708603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:34.709463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:34.714029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 78/168] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:34.714965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7476c87-ea8a-47ba-8d02-464c38e167f1 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:34.749901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea2eabd-dc40-4d8b-a4b8-37753c6d2169 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:34.773257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:34.818670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79733d4a-0db5-414d-bdb7-327ff970e591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:34.819289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79733d4a-0db5-414d-bdb7-327ff970e591 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:34.819570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79733d4a-0db5-414d-bdb7-327ff970e591 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:34.819846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79733d4a-0db5-414d-bdb7-327ff970e591 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:34.822967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7476c87-ea8a-47ba-8d02-464c38e167f1 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:34.877885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99764048-b17c-4f4d-bbaf-393eae15c045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:34.879040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99764048-b17c-4f4d-bbaf-393eae15c045 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:34.881116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99764048-b17c-4f4d-bbaf-393eae15c045 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:34.881655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99764048-b17c-4f4d-bbaf-393eae15c045 None admin] PATH_INFO: `/v3/users/2407f024efda4194a4845b435b0ac7b0` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:34.883709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99764048-b17c-4f4d-bbaf-393eae15c045 None admin] RBAC: Authorizing `identity:get_user(user_id=2407f024efda4194a4845b435b0ac7b0)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:34.887304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99764048-b17c-4f4d-bbaf-393eae15c045 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:34.891037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 79/169] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:34 2022] GET /identity/v3/users/2407f024efda4194a4845b435b0ac7b0 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:34.901670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99764048-b17c-4f4d-bbaf-393eae15c045 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:34.925308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:34.967623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79733d4a-0db5-414d-bdb7-327ff970e591 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:34.969189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79733d4a-0db5-414d-bdb7-327ff970e591 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:35.099204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99764048-b17c-4f4d-bbaf-393eae15c045 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:35.146895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:35.157396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb041f-b2eb-4153-83a2-9de7c12da960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:35.158398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb041f-b2eb-4153-83a2-9de7c12da960 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:35.158886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb041f-b2eb-4153-83a2-9de7c12da960 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:35.159344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb041f-b2eb-4153-83a2-9de7c12da960 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:35.160223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb041f-b2eb-4153-83a2-9de7c12da960 None admin] RBAC: Adding query filter params (name=member) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:35.160730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb041f-b2eb-4153-83a2-9de7c12da960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:35.162360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb041f-b2eb-4153-83a2-9de7c12da960 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:35.167970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 91/170] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:35.168852 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79733d4a-0db5-414d-bdb7-327ff970e591 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:35.172879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 80/171] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:35.179690 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb041f-b2eb-4153-83a2-9de7c12da960 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:35.311242 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:35.363445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79733d4a-0db5-414d-bdb7-327ff970e591 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:35.399728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb041f-b2eb-4153-83a2-9de7c12da960 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:35.422269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ad1ba-d4f7-4368-b929-5a1e124a0d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:35.423210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ad1ba-d4f7-4368-b929-5a1e124a0d3a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:35.423679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ad1ba-d4f7-4368-b929-5a1e124a0d3a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:35.424145 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ad1ba-d4f7-4368-b929-5a1e124a0d3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:35.443882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f67ecf-8109-4a6b-975c-0b1ddeaafa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:35.444519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f67ecf-8109-4a6b-975c-0b1ddeaafa3b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:35.444856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f67ecf-8109-4a6b-975c-0b1ddeaafa3b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:35.445120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f67ecf-8109-4a6b-975c-0b1ddeaafa3b None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:35.446337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f67ecf-8109-4a6b-975c-0b1ddeaafa3b None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:35.449090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f67ecf-8109-4a6b-975c-0b1ddeaafa3b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:35.450804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 81/172] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:35 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:35.457510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f67ecf-8109-4a6b-975c-0b1ddeaafa3b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:35.545302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:35.559835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ad1ba-d4f7-4368-b929-5a1e124a0d3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:35.561312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ad1ba-d4f7-4368-b929-5a1e124a0d3a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:35.605024 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f67ecf-8109-4a6b-975c-0b1ddeaafa3b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:35.642056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8829d5-fe82-47f4-b8d6-eed232cb46b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:35.642932 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8829d5-fe82-47f4-b8d6-eed232cb46b8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:35.647354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8829d5-fe82-47f4-b8d6-eed232cb46b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:35.647354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8829d5-fe82-47f4-b8d6-eed232cb46b8 None admin] PATH_INFO: `/v3/users/2407f024efda4194a4845b435b0ac7b0` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:35.647354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8829d5-fe82-47f4-b8d6-eed232cb46b8 None admin] RBAC: Authorizing `identity:get_user(user_id=2407f024efda4194a4845b435b0ac7b0)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:35.647354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8829d5-fe82-47f4-b8d6-eed232cb46b8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:35.648642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 82/173] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:35 2022] GET /identity/v3/users/2407f024efda4194a4845b435b0ac7b0 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:35.654784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75565-2add-4954-92e8-c5e39f3b7540 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:35.655245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75565-2add-4954-92e8-c5e39f3b7540 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:35.655682 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75565-2add-4954-92e8-c5e39f3b7540 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:35.657230 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 83/174] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:35.658446 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc75565-2add-4954-92e8-c5e39f3b7540 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:35.763043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:35.773345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc75565-2add-4954-92e8-c5e39f3b7540 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:35.786776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 92/175] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:35.787649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ad1ba-d4f7-4368-b929-5a1e124a0d3a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:35.796746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:35.822548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef39a7-3a56-4660-93fd-c1810fad5fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:35.823497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef39a7-3a56-4660-93fd-c1810fad5fa4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:35.824100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef39a7-3a56-4660-93fd-c1810fad5fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:35.824372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef39a7-3a56-4660-93fd-c1810fad5fa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:35.825430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef39a7-3a56-4660-93fd-c1810fad5fa4 None admin] RBAC: Adding query filter params (role.id=20d35b10934447938237c0db5202bbfb, scope.project.id=28843c17667d4709ab63a1378bc693a1, user.id=2407f024efda4194a4845b435b0ac7b0) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:35.825928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef39a7-3a56-4660-93fd-c1810fad5fa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:35.828115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef39a7-3a56-4660-93fd-c1810fad5fa4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:35.836362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 84/176] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:35 2022] GET /identity/v3/role_assignments?user.id=2407f024efda4194a4845b435b0ac7b0&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=20d35b10934447938237c0db5202bbfb => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:35.838164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a42ea-3a70-444d-8e0b-39f2fdca2b1e None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:35.838698 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a42ea-3a70-444d-8e0b-39f2fdca2b1e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:35.838977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8a42ea-3a70-444d-8e0b-39f2fdca2b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:35.864147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ec8a42ea-3a70-444d-8e0b-39f2fdca2b1e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:35.866743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ec8a42ea-3a70-444d-8e0b-39f2fdca2b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:35.909160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ad1ba-d4f7-4368-b929-5a1e124a0d3a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:35.950793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07567e3-20a1-4d7c-a09c-61182479be94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:35.951437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07567e3-20a1-4d7c-a09c-61182479be94 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:35.951713 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07567e3-20a1-4d7c-a09c-61182479be94 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:35.951977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07567e3-20a1-4d7c-a09c-61182479be94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:35.959885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 85/177] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:35.960893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369d06-bbd2-4293-81c1-9c85c98a7eb9 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:35.961179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369d06-bbd2-4293-81c1-9c85c98a7eb9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:35.961449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6369d06-bbd2-4293-81c1-9c85c98a7eb9 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:35.967419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 86/178] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:35.969069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6369d06-bbd2-4293-81c1-9c85c98a7eb9 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:36.024334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:36.050414 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07567e3-20a1-4d7c-a09c-61182479be94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:36.051918 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07567e3-20a1-4d7c-a09c-61182479be94 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:36.093236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6369d06-bbd2-4293-81c1-9c85c98a7eb9 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:36.105591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:36.128476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f09c35-26bb-4630-af73-b90c39e0a7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:36.129105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f09c35-26bb-4630-af73-b90c39e0a7a8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.129381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f09c35-26bb-4630-af73-b90c39e0a7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.129653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f09c35-26bb-4630-af73-b90c39e0a7a8 None admin] PATH_INFO: `/v3/projects/724cf809db59488aa5a4aaa19db80a11` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.130835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f09c35-26bb-4630-af73-b90c39e0a7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=724cf809db59488aa5a4aaa19db80a11)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:36.132262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f09c35-26bb-4630-af73-b90c39e0a7a8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:36.133877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 87/179] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:35 2022] GET /identity/v3/projects/724cf809db59488aa5a4aaa19db80a11 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:36.135076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3580c06-921e-414e-b09d-4fb6fb025f40 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.135386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3580c06-921e-414e-b09d-4fb6fb025f40 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.135662 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3580c06-921e-414e-b09d-4fb6fb025f40 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.138234 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 88/180] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:36.140168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e67bd-99af-46d9-a1fb-f53550abfa83 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.140452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e67bd-99af-46d9-a1fb-f53550abfa83 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.140729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e67bd-99af-46d9-a1fb-f53550abfa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.161669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-a12e67bd-99af-46d9-a1fb-f53550abfa83 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:36.161886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 93/181] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:36.162950 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a12e67bd-99af-46d9-a1fb-f53550abfa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:36.163172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c790-8f2a-41df-b0cc-27a7618dbb9a None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.163519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c790-8f2a-41df-b0cc-27a7618dbb9a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.163796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29c790-8f2a-41df-b0cc-27a7618dbb9a None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.166868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 94/182] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:36.166868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29c790-8f2a-41df-b0cc-27a7618dbb9a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:36.232026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 89/183] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:36.233097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e67bd-99af-46d9-a1fb-f53550abfa83 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:36.262978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:36.282656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29c790-8f2a-41df-b0cc-27a7618dbb9a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:36.317920 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9dbb1-465a-4a9f-9d95-402556583359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:36.318560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9dbb1-465a-4a9f-9d95-402556583359 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.318836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9dbb1-465a-4a9f-9d95-402556583359 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.319104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9dbb1-465a-4a9f-9d95-402556583359 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.328901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9dbb1-465a-4a9f-9d95-402556583359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:36.332290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e67bd-99af-46d9-a1fb-f53550abfa83 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-42c9dbb1-465a-4a9f-9d95-402556583359 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:36.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:36.335178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:36.335945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 95/184] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:31:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:36.336600 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89863-58fd-4460-8b0f-70461de66a3e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.337281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89863-58fd-4460-8b0f-70461de66a3e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.337281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89863-58fd-4460-8b0f-70461de66a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.343290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:36.354727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-36c89863-58fd-4460-8b0f-70461de66a3e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:36.365167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-36c89863-58fd-4460-8b0f-70461de66a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:36.367284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681fe88-bcca-4e41-ab5b-c83e2aef1974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:36.367877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681fe88-bcca-4e41-ab5b-c83e2aef1974 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.368149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681fe88-bcca-4e41-ab5b-c83e2aef1974 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.368411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681fe88-bcca-4e41-ab5b-c83e2aef1974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.369127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681fe88-bcca-4e41-ab5b-c83e2aef1974 None admin] RBAC: Adding query filter params (role.id=2490390ea61f4cf594df3587e7674650, scope.project.id=724cf809db59488aa5a4aaa19db80a11, user.id=2407f024efda4194a4845b435b0ac7b0) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:36.369485 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681fe88-bcca-4e41-ab5b-c83e2aef1974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:36.370865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681fe88-bcca-4e41-ab5b-c83e2aef1974 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:36.376425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 90/185] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:36 2022] GET /identity/v3/role_assignments?user.id=2407f024efda4194a4845b435b0ac7b0&scope.project.id=724cf809db59488aa5a4aaa19db80a11&role.id=2490390ea61f4cf594df3587e7674650 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:36.377192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681fe88-bcca-4e41-ab5b-c83e2aef1974 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:36.426416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 96/186] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:36.440394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8b2e-8518-47fa-8cde-1d753005cf2d None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.441023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8b2e-8518-47fa-8cde-1d753005cf2d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.441471 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8b2e-8518-47fa-8cde-1d753005cf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.467513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d6ca8b2e-8518-47fa-8cde-1d753005cf2d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:36.468740 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d6ca8b2e-8518-47fa-8cde-1d753005cf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:36.507067 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681fe88-bcca-4e41-ab5b-c83e2aef1974 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:36.551629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:36.553259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711eb181-639b-4355-b29b-f38c588edcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:36.553259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eb181-639b-4355-b29b-f38c588edcbb None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.553798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eb181-639b-4355-b29b-f38c588edcbb None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.553798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711eb181-639b-4355-b29b-f38c588edcbb None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.554822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711eb181-639b-4355-b29b-f38c588edcbb None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:36.556313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711eb181-639b-4355-b29b-f38c588edcbb None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:36.556720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 97/187] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:36.557348 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca8b2e-8518-47fa-8cde-1d753005cf2d None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:36.558368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 91/188] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:36 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:36.559070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711eb181-639b-4355-b29b-f38c588edcbb None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:36.608834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:36.669304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711eb181-639b-4355-b29b-f38c588edcbb None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:36.671550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca8b2e-8518-47fa-8cde-1d753005cf2d None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:36.704203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a20aff-3686-4f57-aa17-f2c1ad39770f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:36.704396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa6544-0762-4627-828c-cbee2677e4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:36.704823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a20aff-3686-4f57-aa17-f2c1ad39770f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.704923 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6544-0762-4627-828c-cbee2677e4cf None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.705096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a20aff-3686-4f57-aa17-f2c1ad39770f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.705180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6544-0762-4627-828c-cbee2677e4cf None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.705345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a20aff-3686-4f57-aa17-f2c1ad39770f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.705431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa6544-0762-4627-828c-cbee2677e4cf None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.706061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6544-0762-4627-828c-cbee2677e4cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:36.706357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6544-0762-4627-828c-cbee2677e4cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:36.707712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa6544-0762-4627-828c-cbee2677e4cf None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:36.715653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a20aff-3686-4f57-aa17-f2c1ad39770f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:36.716029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 92/189] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:36.717976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa6544-0762-4627-828c-cbee2677e4cf None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-60a20aff-3686-4f57-aa17-f2c1ad39770f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:36.720574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:36.721446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 98/190] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:31:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:36.722630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a20aff-3686-4f57-aa17-f2c1ad39770f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:36.782837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:36.817829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa6544-0762-4627-828c-cbee2677e4cf None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:36.819390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:36.822669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a20aff-3686-4f57-aa17-f2c1ad39770f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:36.853354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9261e-7a67-485f-91d4-b25d90788dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:36.854163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9261e-7a67-485f-91d4-b25d90788dfa None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.854592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9261e-7a67-485f-91d4-b25d90788dfa None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.854976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9261e-7a67-485f-91d4-b25d90788dfa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.857725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2869a-1c40-41fa-9756-1e4377845331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:36.858347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2869a-1c40-41fa-9756-1e4377845331 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:36.858621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2869a-1c40-41fa-9756-1e4377845331 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:36.858886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2869a-1c40-41fa-9756-1e4377845331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:36.859601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2869a-1c40-41fa-9756-1e4377845331 None admin] RBAC: Adding query filter params (role.id=2490390ea61f4cf594df3587e7674650, scope.project.id=28843c17667d4709ab63a1378bc693a1, user.id=2407f024efda4194a4845b435b0ac7b0) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:36.859899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2869a-1c40-41fa-9756-1e4377845331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:36.861325 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2869a-1c40-41fa-9756-1e4377845331 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:36.866215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9261e-7a67-485f-91d4-b25d90788dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:36.867027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 99/191] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:36 2022] GET /identity/v3/role_assignments?user.id=2407f024efda4194a4845b435b0ac7b0&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=2490390ea61f4cf594df3587e7674650 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:36.870891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2869a-1c40-41fa-9756-1e4377845331 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-5cb9261e-7a67-485f-91d4-b25d90788dfa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:36.871369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:36.872256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:36.873040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 93/192] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:36.873686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9261e-7a67-485f-91d4-b25d90788dfa None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:36.992064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2869a-1c40-41fa-9756-1e4377845331 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:36.996128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9261e-7a67-485f-91d4-b25d90788dfa None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:37.017265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:37.032542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2fac9-0e1e-45cb-903f-fb31e7431a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.033267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2fac9-0e1e-45cb-903f-fb31e7431a8a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.044127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2fac9-0e1e-45cb-903f-fb31e7431a8a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.044127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2fac9-0e1e-45cb-903f-fb31e7431a8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:37.044127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2fac9-0e1e-45cb-903f-fb31e7431a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:37.044382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e5489-ca4c-4d96-88ed-4b9c48822c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.044382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e5489-ca4c-4d96-88ed-4b9c48822c49 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.044558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2fac9-0e1e-45cb-903f-fb31e7431a8a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:37.044639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e5489-ca4c-4d96-88ed-4b9c48822c49 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.044639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e5489-ca4c-4d96-88ed-4b9c48822c49 None admin] PATH_INFO: `/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:37.044800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 100/193] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:37.044800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2fac9-0e1e-45cb-903f-fb31e7431a8a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:37.061376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e5489-ca4c-4d96-88ed-4b9c48822c49 None admin] RBAC: Authorizing `identity:get_user(user_id=d31d7aa5563f4253b885ce8ee7bbcb47)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:37.063113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e5489-ca4c-4d96-88ed-4b9c48822c49 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:37.064841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 94/194] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:36 2022] GET /identity/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:37.065628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e5489-ca4c-4d96-88ed-4b9c48822c49 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:37.090758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:37.149903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2fac9-0e1e-45cb-903f-fb31e7431a8a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:37.172143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e5489-ca4c-4d96-88ed-4b9c48822c49 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:37.183876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fcd52-1b60-41f0-ae75-f9baf57228aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.184478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fcd52-1b60-41f0-ae75-f9baf57228aa None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.184770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fcd52-1b60-41f0-ae75-f9baf57228aa None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.185151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fcd52-1b60-41f0-ae75-f9baf57228aa None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:37.185889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fcd52-1b60-41f0-ae75-f9baf57228aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:37.186237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fcd52-1b60-41f0-ae75-f9baf57228aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:37.187780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fcd52-1b60-41f0-ae75-f9baf57228aa None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:37.196642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 101/195] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:31:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:37.196642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fcd52-1b60-41f0-ae75-f9baf57228aa None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:37.207362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409dd7a-e786-4b08-8d01-ade24373bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.208002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409dd7a-e786-4b08-8d01-ade24373bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.208792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409dd7a-e786-4b08-8d01-ade24373bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.209056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409dd7a-e786-4b08-8d01-ade24373bdf9 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:37.209771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409dd7a-e786-4b08-8d01-ade24373bdf9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:37.210090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409dd7a-e786-4b08-8d01-ade24373bdf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:37.211446 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409dd7a-e786-4b08-8d01-ade24373bdf9 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:37.220126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 95/196] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:37.222174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409dd7a-e786-4b08-8d01-ade24373bdf9 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:37.255644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:37.298660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fcd52-1b60-41f0-ae75-f9baf57228aa None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:37.302927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:37.321503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409dd7a-e786-4b08-8d01-ade24373bdf9 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:37.333667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae35d30-3dcd-4237-8609-628d78f203d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.334293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae35d30-3dcd-4237-8609-628d78f203d0 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.334568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae35d30-3dcd-4237-8609-628d78f203d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.334833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae35d30-3dcd-4237-8609-628d78f203d0 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:37.335503 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae35d30-3dcd-4237-8609-628d78f203d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:37.336933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae35d30-3dcd-4237-8609-628d78f203d0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:37.359947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23654157-a194-40c7-92f7-9c3171cf53b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.359947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23654157-a194-40c7-92f7-9c3171cf53b9 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.359947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23654157-a194-40c7-92f7-9c3171cf53b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.359947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23654157-a194-40c7-92f7-9c3171cf53b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:37.422277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23654157-a194-40c7-92f7-9c3171cf53b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:37.423728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23654157-a194-40c7-92f7-9c3171cf53b9 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:37.445447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 102/197] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:31:37 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:37.446422 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae35d30-3dcd-4237-8609-628d78f203d0 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:37.533158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:37.556483 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 96/198] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:37.557360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23654157-a194-40c7-92f7-9c3171cf53b9 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:37.576014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae35d30-3dcd-4237-8609-628d78f203d0 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:37.587903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:37.616749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0808-5ceb-4de0-876c-4f7cef435a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.617300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0808-5ceb-4de0-876c-4f7cef435a0b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.617578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0808-5ceb-4de0-876c-4f7cef435a0b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.617848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb0808-5ceb-4de0-876c-4f7cef435a0b None admin] PATH_INFO: `/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:37.619525 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb0808-5ceb-4de0-876c-4f7cef435a0b None admin] RBAC: Authorizing `identity:get_user(user_id=d31d7aa5563f4253b885ce8ee7bbcb47)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:37.620967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb0808-5ceb-4de0-876c-4f7cef435a0b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:37.623335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 103/199] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:37 2022] GET /identity/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:37.624133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0808-5ceb-4de0-876c-4f7cef435a0b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:37.680656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23654157-a194-40c7-92f7-9c3171cf53b9 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:37.723621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28660f59-fea9-4f36-adba-527f67ee9bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.724608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28660f59-fea9-4f36-adba-527f67ee9bf7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.724883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28660f59-fea9-4f36-adba-527f67ee9bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.725146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28660f59-fea9-4f36-adba-527f67ee9bf7 None admin] PATH_INFO: `/v3/users/0f2132533f50466da1e02d5503f6d57c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:37.726632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28660f59-fea9-4f36-adba-527f67ee9bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=0f2132533f50466da1e02d5503f6d57c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:37.728129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28660f59-fea9-4f36-adba-527f67ee9bf7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:37.730619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 97/200] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:37 2022] GET /identity/v3/users/0f2132533f50466da1e02d5503f6d57c => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:37.731317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28660f59-fea9-4f36-adba-527f67ee9bf7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:37.753902 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb0808-5ceb-4de0-876c-4f7cef435a0b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:37.794091 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:37.796305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f8620-ea00-4cd5-9dea-a14e0c47108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.796942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f8620-ea00-4cd5-9dea-a14e0c47108a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.797220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f8620-ea00-4cd5-9dea-a14e0c47108a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.797477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f8620-ea00-4cd5-9dea-a14e0c47108a None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:37.798687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f8620-ea00-4cd5-9dea-a14e0c47108a None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:37.800138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f8620-ea00-4cd5-9dea-a14e0c47108a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:37.803045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 104/201] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:37 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:37.803899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f8620-ea00-4cd5-9dea-a14e0c47108a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:37.854332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:37.855269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28660f59-fea9-4f36-adba-527f67ee9bf7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:37.906404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e703e-64f8-4999-8efd-6d6d68d788d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.907946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e703e-64f8-4999-8efd-6d6d68d788d6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.908408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e703e-64f8-4999-8efd-6d6d68d788d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.909089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e703e-64f8-4999-8efd-6d6d68d788d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:37.915124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f8620-ea00-4cd5-9dea-a14e0c47108a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:37.953748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4517ab-3f8a-4f62-a717-45f7da906fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:37.954547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4517ab-3f8a-4f62-a717-45f7da906fe1 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:37.954895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4517ab-3f8a-4f62-a717-45f7da906fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:37.955240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4517ab-3f8a-4f62-a717-45f7da906fe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:38.001258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e703e-64f8-4999-8efd-6d6d68d788d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:38.002931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e703e-64f8-4999-8efd-6d6d68d788d6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:38.041826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4517ab-3f8a-4f62-a717-45f7da906fe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:38.043657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4517ab-3f8a-4f62-a717-45f7da906fe1 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:38.055694 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:38.093514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:38.110581 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 98/202] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:38.111427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e703e-64f8-4999-8efd-6d6d68d788d6 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:38.151806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 105/203] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:38.153015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4517ab-3f8a-4f62-a717-45f7da906fe1 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:38.219445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e703e-64f8-4999-8efd-6d6d68d788d6 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:38.257773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b434a5a-585d-42ff-bee3-ac2c25afabca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:38.258507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b434a5a-585d-42ff-bee3-ac2c25afabca None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:38.259040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b434a5a-585d-42ff-bee3-ac2c25afabca None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:38.259147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b434a5a-585d-42ff-bee3-ac2c25afabca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:38.259940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b434a5a-585d-42ff-bee3-ac2c25afabca None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=5796807ec0ad4cab845934a9df6b9689, user.id=d31d7aa5563f4253b885ce8ee7bbcb47) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:38.260323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b434a5a-585d-42ff-bee3-ac2c25afabca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:38.261890 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b434a5a-585d-42ff-bee3-ac2c25afabca None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:38.262643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4517ab-3f8a-4f62-a717-45f7da906fe1 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:38.268059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 99/204] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:38 2022] GET /identity/v3/role_assignments?user.id=d31d7aa5563f4253b885ce8ee7bbcb47&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:38.268959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b434a5a-585d-42ff-bee3-ac2c25afabca None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:38.287169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:38.315091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b963d1-6844-41c7-ac19-2364a8c20ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:38.316112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b963d1-6844-41c7-ac19-2364a8c20ebe None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:38.316583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b963d1-6844-41c7-ac19-2364a8c20ebe None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:38.317076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b963d1-6844-41c7-ac19-2364a8c20ebe None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:38.318533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b963d1-6844-41c7-ac19-2364a8c20ebe None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:38.320203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b963d1-6844-41c7-ac19-2364a8c20ebe None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:38.322057 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 106/205] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:38 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:38.328238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b963d1-6844-41c7-ac19-2364a8c20ebe None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:38.354715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:38.385830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b434a5a-585d-42ff-bee3-ac2c25afabca None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:38.425701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1b484-131e-49dd-a315-87d26deac18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:38.426481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1b484-131e-49dd-a315-87d26deac18b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:38.426859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1b484-131e-49dd-a315-87d26deac18b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:38.427200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1b484-131e-49dd-a315-87d26deac18b None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:38.428607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1b484-131e-49dd-a315-87d26deac18b None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:38.430367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1b484-131e-49dd-a315-87d26deac18b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:38.432412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 100/206] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:38 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:38.438136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1b484-131e-49dd-a315-87d26deac18b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:38.450191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b963d1-6844-41c7-ac19-2364a8c20ebe None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:38.502765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdfcdd-1f35-4bd2-a640-50ac62e45600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:38.503689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdfcdd-1f35-4bd2-a640-50ac62e45600 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:38.504099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdfcdd-1f35-4bd2-a640-50ac62e45600 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:38.504545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdfcdd-1f35-4bd2-a640-50ac62e45600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:38.505886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdfcdd-1f35-4bd2-a640-50ac62e45600 None admin] RBAC: Adding query filter params (role.id=20d35b10934447938237c0db5202bbfb, scope.project.id=5796807ec0ad4cab845934a9df6b9689, user.id=d31d7aa5563f4253b885ce8ee7bbcb47) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:38.506362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdfcdd-1f35-4bd2-a640-50ac62e45600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:38.508043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdfcdd-1f35-4bd2-a640-50ac62e45600 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:38.522531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 107/207] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:38 2022] GET /identity/v3/role_assignments?user.id=d31d7aa5563f4253b885ce8ee7bbcb47&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=20d35b10934447938237c0db5202bbfb => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:38.535131 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:38.558478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1b484-131e-49dd-a315-87d26deac18b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:38.599504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d904c-c2ec-4c42-8e01-f0ffdc85a3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:38.600132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d904c-c2ec-4c42-8e01-f0ffdc85a3a0 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:38.600407 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d904c-c2ec-4c42-8e01-f0ffdc85a3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:38.600674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d904c-c2ec-4c42-8e01-f0ffdc85a3a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:38.601412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d904c-c2ec-4c42-8e01-f0ffdc85a3a0 None admin] RBAC: Adding query filter params (role.id=d7c03cc554a74d0aad88976354649d2d, scope.project.id=28843c17667d4709ab63a1378bc693a1, user.id=0f2132533f50466da1e02d5503f6d57c) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:38.601715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d904c-c2ec-4c42-8e01-f0ffdc85a3a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:38.603222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d904c-c2ec-4c42-8e01-f0ffdc85a3a0 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:38.610497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 101/208] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:38 2022] GET /identity/v3/role_assignments?user.id=0f2132533f50466da1e02d5503f6d57c&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=d7c03cc554a74d0aad88976354649d2d => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:39.713409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06681e22-4421-4022-98f2-97e1530cd8b8 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:39.713928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06681e22-4421-4022-98f2-97e1530cd8b8 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:39.713928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06681e22-4421-4022-98f2-97e1530cd8b8 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:39.715462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 108/209] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:39.720972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b2cba-c815-44b3-8e02-1f3eef6fd394 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:39.721818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b2cba-c815-44b3-8e02-1f3eef6fd394 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:39.722315 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b2cba-c815-44b3-8e02-1f3eef6fd394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:39.741364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-a36b2cba-c815-44b3-8e02-1f3eef6fd394 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:39.742905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a36b2cba-c815-44b3-8e02-1f3eef6fd394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:39.813639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f48e82-98c8-4b52-8176-981052e2d5c7 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:39.813917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f48e82-98c8-4b52-8176-981052e2d5c7 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:39.814111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f48e82-98c8-4b52-8176-981052e2d5c7 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:39.816001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 109/210] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:39.821760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 102/211] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:39.823155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65d427-da79-45f7-b062-b2a66a9270ff None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:39.823436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65d427-da79-45f7-b062-b2a66a9270ff None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:39.823705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65d427-da79-45f7-b062-b2a66a9270ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:39.842589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b2cba-c815-44b3-8e02-1f3eef6fd394 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:39.850954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ce65d427-da79-45f7-b062-b2a66a9270ff None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:39.853029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ce65d427-da79-45f7-b062-b2a66a9270ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:39.924856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:39.949391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b2cba-c815-44b3-8e02-1f3eef6fd394 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:39.952943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 110/212] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:39.981230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65d427-da79-45f7-b062-b2a66a9270ff None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:39.987761 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cd55d-27aa-445e-94f9-1b5dbe463bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:39.988416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cd55d-27aa-445e-94f9-1b5dbe463bfb None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:39.988833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cd55d-27aa-445e-94f9-1b5dbe463bfb None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:39.989047 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cd55d-27aa-445e-94f9-1b5dbe463bfb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:39.994334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:40.000696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6cd55d-27aa-445e-94f9-1b5dbe463bfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-ee6cd55d-27aa-445e-94f9-1b5dbe463bfb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:40.005541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:40.010459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:40.011261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 103/213] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:31:40.017985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cd55d-27aa-445e-94f9-1b5dbe463bfb None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:40.097447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65d427-da79-45f7-b062-b2a66a9270ff None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:40.119561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cd55d-27aa-445e-94f9-1b5dbe463bfb None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:40.135395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983179-c2f8-4df2-a6d9-b8db300fd14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:40.136032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983179-c2f8-4df2-a6d9-b8db300fd14e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:40.136311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983179-c2f8-4df2-a6d9-b8db300fd14e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:40.136581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983179-c2f8-4df2-a6d9-b8db300fd14e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:40.137707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b983179-c2f8-4df2-a6d9-b8db300fd14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:40.139293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b983179-c2f8-4df2-a6d9-b8db300fd14e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:40.146249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 111/214] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:40.147714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983179-c2f8-4df2-a6d9-b8db300fd14e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:40.160902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65204e3-1da9-4e67-a474-c6dd405d2c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:40.161738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65204e3-1da9-4e67-a474-c6dd405d2c00 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:40.162220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65204e3-1da9-4e67-a474-c6dd405d2c00 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:40.162628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65204e3-1da9-4e67-a474-c6dd405d2c00 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:40.163536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65204e3-1da9-4e67-a474-c6dd405d2c00 None admin] RBAC: Adding query filter params (name=member) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:40.163967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65204e3-1da9-4e67-a474-c6dd405d2c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:40.165519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65204e3-1da9-4e67-a474-c6dd405d2c00 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:40.168231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:40.175411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 104/215] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:40.182491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65204e3-1da9-4e67-a474-c6dd405d2c00 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:40.240136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:40.268014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983179-c2f8-4df2-a6d9-b8db300fd14e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:40.294380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65204e3-1da9-4e67-a474-c6dd405d2c00 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:40.308744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad317e9-92d8-41cb-bfee-57245ff07f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:40.309352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad317e9-92d8-41cb-bfee-57245ff07f44 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:40.309635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad317e9-92d8-41cb-bfee-57245ff07f44 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:40.309937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad317e9-92d8-41cb-bfee-57245ff07f44 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:40.310697 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad317e9-92d8-41cb-bfee-57245ff07f44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:40.312255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad317e9-92d8-41cb-bfee-57245ff07f44 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:40.333645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1018938-440e-4828-b141-cd9f27e9dad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:40.334774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1018938-440e-4828-b141-cd9f27e9dad7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:40.335249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1018938-440e-4828-b141-cd9f27e9dad7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:40.335692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1018938-440e-4828-b141-cd9f27e9dad7 None admin] PATH_INFO: `/v3/users/6717eba732714a84bdfd052cf5aceb40` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:40.351597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1018938-440e-4828-b141-cd9f27e9dad7 None admin] RBAC: Authorizing `identity:get_user(user_id=6717eba732714a84bdfd052cf5aceb40)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:40.353394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1018938-440e-4828-b141-cd9f27e9dad7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:40.360222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 105/216] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:40 2022] GET /identity/v3/users/6717eba732714a84bdfd052cf5aceb40 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:40.361006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1018938-440e-4828-b141-cd9f27e9dad7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:40.410001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 112/217] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:31:40 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:31:40.436845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:40.487873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1018938-440e-4828-b141-cd9f27e9dad7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:40.535558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b2504-4c3d-488f-98bb-c1087ced1dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:40.536463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b2504-4c3d-488f-98bb-c1087ced1dc8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:40.536832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b2504-4c3d-488f-98bb-c1087ced1dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:40.537125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b2504-4c3d-488f-98bb-c1087ced1dc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:40.590566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7a1ca-8ca2-4b71-a224-6ce9ed96b35a None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:40.591162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7a1ca-8ca2-4b71-a224-6ce9ed96b35a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:40.591631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7a1ca-8ca2-4b71-a224-6ce9ed96b35a None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:40.593159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 113/218] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:31:40.598707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01327741-7e9d-4011-a195-e08cd2dec300 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:40.598990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01327741-7e9d-4011-a195-e08cd2dec300 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:40.599264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01327741-7e9d-4011-a195-e08cd2dec300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:40.616981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b2504-4c3d-488f-98bb-c1087ced1dc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:40.618392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b2504-4c3d-488f-98bb-c1087ced1dc8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:40.618719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-01327741-7e9d-4011-a195-e08cd2dec300 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:40.620137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-01327741-7e9d-4011-a195-e08cd2dec300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:40.715204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:40.733599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 114/219] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:40.735031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b09d6-c5b0-4b73-bc2a-e39aeafb0a42 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:40.735333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b09d6-c5b0-4b73-bc2a-e39aeafb0a42 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:40.735601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b09d6-c5b0-4b73-bc2a-e39aeafb0a42 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:40.737114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 115/220] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:40.743104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca3784-2f8e-4ea5-88eb-488993f2ef43 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:40.743383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca3784-2f8e-4ea5-88eb-488993f2ef43 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:40.743658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca3784-2f8e-4ea5-88eb-488993f2ef43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:40.764874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ecca3784-2f8e-4ea5-88eb-488993f2ef43 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:40.766906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ecca3784-2f8e-4ea5-88eb-488993f2ef43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:40.781859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 106/221] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 16:31:40.783845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b2504-4c3d-488f-98bb-c1087ced1dc8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:40.858640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:40.872064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 116/222] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:40.872064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca3784-2f8e-4ea5-88eb-488993f2ef43 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:40.920413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b2504-4c3d-488f-98bb-c1087ced1dc8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:40.968648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6df459-963a-4cbf-a443-8b5bab30a06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:40.969283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6df459-963a-4cbf-a443-8b5bab30a06f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:40.969557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6df459-963a-4cbf-a443-8b5bab30a06f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:40.970260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6df459-963a-4cbf-a443-8b5bab30a06f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:40.984335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca3784-2f8e-4ea5-88eb-488993f2ef43 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:40.991054 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6df459-963a-4cbf-a443-8b5bab30a06f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-4f6df459-963a-4cbf-a443-8b5bab30a06f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:40.997075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:40.997975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:40.998805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:41.001601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 107/223] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:31:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:41.002912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6df459-963a-4cbf-a443-8b5bab30a06f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:41.026959 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f1115-fbfd-4fcd-87a0-913b2d01cfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:41.027879 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f1115-fbfd-4fcd-87a0-913b2d01cfa5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:41.028237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:41.028349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f1115-fbfd-4fcd-87a0-913b2d01cfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:41.028738 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f1115-fbfd-4fcd-87a0-913b2d01cfa5 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:41.030697 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f1115-fbfd-4fcd-87a0-913b2d01cfa5 None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:41.032330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150f1115-fbfd-4fcd-87a0-913b2d01cfa5 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:41.034495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 117/224] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:40 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:41.040090 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f1115-fbfd-4fcd-87a0-913b2d01cfa5 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:41.112555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:41.128718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6df459-963a-4cbf-a443-8b5bab30a06f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:41.159277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150f1115-fbfd-4fcd-87a0-913b2d01cfa5 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:41.165392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce51bf-7310-42c7-8d92-0f64b815fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:41.166057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce51bf-7310-42c7-8d92-0f64b815fe6b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:41.166338 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce51bf-7310-42c7-8d92-0f64b815fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:41.166599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce51bf-7310-42c7-8d92-0f64b815fe6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:41.178340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce51bf-7310-42c7-8d92-0f64b815fe6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-71ce51bf-7310-42c7-8d92-0f64b815fe6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:41.183317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:41.184208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:41.185029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 108/225] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:31:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:41.185687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce51bf-7310-42c7-8d92-0f64b815fe6b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:41.203317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af89ae-5fb9-4eb0-8401-5ee81df4c0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:41.204257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af89ae-5fb9-4eb0-8401-5ee81df4c0ca None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:41.204831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af89ae-5fb9-4eb0-8401-5ee81df4c0ca None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:41.205302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af89ae-5fb9-4eb0-8401-5ee81df4c0ca None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:41.206318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af89ae-5fb9-4eb0-8401-5ee81df4c0ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:41.206868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af89ae-5fb9-4eb0-8401-5ee81df4c0ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:41.209110 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af89ae-5fb9-4eb0-8401-5ee81df4c0ca None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:41.219580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 118/226] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:41.220771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af89ae-5fb9-4eb0-8401-5ee81df4c0ca None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:41.269701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:41.294852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce51bf-7310-42c7-8d92-0f64b815fe6b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:41.330964 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98488b53-e9d9-40d4-9e60-68e546512766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:41.331605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488b53-e9d9-40d4-9e60-68e546512766 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:41.331892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488b53-e9d9-40d4-9e60-68e546512766 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:41.332163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488b53-e9d9-40d4-9e60-68e546512766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:41.333408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98488b53-e9d9-40d4-9e60-68e546512766 None admin] RBAC: Adding query filter params (role.id=2490390ea61f4cf594df3587e7674650, scope.project.id=5796807ec0ad4cab845934a9df6b9689, user.id=6717eba732714a84bdfd052cf5aceb40) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:41.333712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98488b53-e9d9-40d4-9e60-68e546512766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:41.335240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98488b53-e9d9-40d4-9e60-68e546512766 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:41.336186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af89ae-5fb9-4eb0-8401-5ee81df4c0ca None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:41.348622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 109/227] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:41 2022] GET /identity/v3/role_assignments?user.id=6717eba732714a84bdfd052cf5aceb40&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=2490390ea61f4cf594df3587e7674650 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:41.348622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98488b53-e9d9-40d4-9e60-68e546512766 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:41.382531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:41.404961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb986625-e2dd-48bd-811a-630236a4b4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:41.405940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb986625-e2dd-48bd-811a-630236a4b4ff None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:41.406469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb986625-e2dd-48bd-811a-630236a4b4ff None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:41.406935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb986625-e2dd-48bd-811a-630236a4b4ff None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:41.407894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb986625-e2dd-48bd-811a-630236a4b4ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:41.408443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb986625-e2dd-48bd-811a-630236a4b4ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:41.410142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb986625-e2dd-48bd-811a-630236a4b4ff None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:41.433466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 119/228] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:31:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:41.441631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb986625-e2dd-48bd-811a-630236a4b4ff None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:41.459946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98488b53-e9d9-40d4-9e60-68e546512766 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:41.510820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fce77-99e4-4425-9543-557b846c21a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:41.511476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fce77-99e4-4425-9543-557b846c21a9 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:41.511753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fce77-99e4-4425-9543-557b846c21a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:41.512056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fce77-99e4-4425-9543-557b846c21a9 None admin] PATH_INFO: `/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:41.518868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fce77-99e4-4425-9543-557b846c21a9 None admin] RBAC: Authorizing `identity:get_user(user_id=d31d7aa5563f4253b885ce8ee7bbcb47)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:41.520899 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84fce77-99e4-4425-9543-557b846c21a9 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:41.522959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 110/229] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:41 2022] GET /identity/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:41.537464 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fce77-99e4-4425-9543-557b846c21a9 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:41.555587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:41.572174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb986625-e2dd-48bd-811a-630236a4b4ff None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:41.634586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b929639-c070-42ba-b699-c9c8b585270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:41.635570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b929639-c070-42ba-b699-c9c8b585270a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:41.636115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b929639-c070-42ba-b699-c9c8b585270a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:41.636581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b929639-c070-42ba-b699-c9c8b585270a None admin] PATH_INFO: `/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:41.639604 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b929639-c070-42ba-b699-c9c8b585270a None admin] RBAC: Authorizing `identity:get_user(user_id=d31d7aa5563f4253b885ce8ee7bbcb47)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:41.645463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b929639-c070-42ba-b699-c9c8b585270a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:41.645873 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 120/230] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:41 2022] GET /identity/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:41.655099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b929639-c070-42ba-b699-c9c8b585270a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:41.731272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fce77-99e4-4425-9543-557b846c21a9 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:41.780272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:41.808359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e297c1c-88aa-4120-bdf8-d6fae1e4910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:41.809734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e297c1c-88aa-4120-bdf8-d6fae1e4910e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:41.810402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e297c1c-88aa-4120-bdf8-d6fae1e4910e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:41.810875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e297c1c-88aa-4120-bdf8-d6fae1e4910e None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:41.812229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e297c1c-88aa-4120-bdf8-d6fae1e4910e None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:41.815276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e297c1c-88aa-4120-bdf8-d6fae1e4910e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:41.819351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 111/231] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:41 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:41.825173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e297c1c-88aa-4120-bdf8-d6fae1e4910e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:41.876133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b929639-c070-42ba-b699-c9c8b585270a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:41.942011 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a30d8-6271-44d3-b739-be65e5eda38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:41.942553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a30d8-6271-44d3-b739-be65e5eda38c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:41.944779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a30d8-6271-44d3-b739-be65e5eda38c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:41.945075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a30d8-6271-44d3-b739-be65e5eda38c None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:41.947132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a30d8-6271-44d3-b739-be65e5eda38c None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:41.949360 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a30d8-6271-44d3-b739-be65e5eda38c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:41.951160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 121/232] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:41 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:41.974735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a30d8-6271-44d3-b739-be65e5eda38c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:41.975388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:42.025287 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e297c1c-88aa-4120-bdf8-d6fae1e4910e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:42.066924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dec5b-a07f-4dfa-9304-622741bf9daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:42.067595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dec5b-a07f-4dfa-9304-622741bf9daa None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:42.067873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dec5b-a07f-4dfa-9304-622741bf9daa None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:42.068145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dec5b-a07f-4dfa-9304-622741bf9daa None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689/users/d31d7aa5563f4253b885ce8ee7bbcb47/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:42.071712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dec5b-a07f-4dfa-9304-622741bf9daa None admin] RBAC: Authorizing `identity:create_grant(project_id=5796807ec0ad4cab845934a9df6b9689, user_id=d31d7aa5563f4253b885ce8ee7bbcb47, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:42.073228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dec5b-a07f-4dfa-9304-622741bf9daa None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:42.142173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 112/233] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:31:41 2022] PUT /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689/users/d31d7aa5563f4253b885ce8ee7bbcb47/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:31:42.144456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a30d8-6271-44d3-b739-be65e5eda38c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:42.168866 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:42.184693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e62ea8-34ff-4060-aa6f-d5afbe9bc71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:42.185369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e62ea8-34ff-4060-aa6f-d5afbe9bc71e None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:42.185647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e62ea8-34ff-4060-aa6f-d5afbe9bc71e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:42.185986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e62ea8-34ff-4060-aa6f-d5afbe9bc71e None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689/users/d31d7aa5563f4253b885ce8ee7bbcb47/roles/20d35b10934447938237c0db5202bbfb` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:42.188427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e62ea8-34ff-4060-aa6f-d5afbe9bc71e None admin] RBAC: Authorizing `identity:create_grant(project_id=5796807ec0ad4cab845934a9df6b9689, user_id=d31d7aa5563f4253b885ce8ee7bbcb47, role_id=20d35b10934447938237c0db5202bbfb)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:42.190068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e62ea8-34ff-4060-aa6f-d5afbe9bc71e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:42.280036 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 122/234] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:31:41 2022] PUT /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689/users/d31d7aa5563f4253b885ce8ee7bbcb47/roles/20d35b10934447938237c0db5202bbfb => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:31:43.100442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c235b-a455-4f8c-a04e-a17ddd666c6c None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:43.100991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c235b-a455-4f8c-a04e-a17ddd666c6c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:43.100991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c235b-a455-4f8c-a04e-a17ddd666c6c None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:43.102629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 113/235] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:43.107834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d700e-7619-4eae-a6a8-7ae2085631c5 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:43.108254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d700e-7619-4eae-a6a8-7ae2085631c5 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:43.108446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d700e-7619-4eae-a6a8-7ae2085631c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:43.134947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ae5d700e-7619-4eae-a6a8-7ae2085631c5 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:43.136142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ae5d700e-7619-4eae-a6a8-7ae2085631c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:43.226808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 123/236] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:43.256655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c235b-a455-4f8c-a04e-a17ddd666c6c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:43.303572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8ec74-04b5-4170-b634-879578cac8b6 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:43.303881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8ec74-04b5-4170-b634-879578cac8b6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:43.304111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8ec74-04b5-4170-b634-879578cac8b6 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:43.305589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 124/237] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:31:43.310664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adc5c9-4653-48f8-894b-300fbae29db1 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:43.310941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adc5c9-4653-48f8-894b-300fbae29db1 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:43.311207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97adc5c9-4653-48f8-894b-300fbae29db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:43.334107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-97adc5c9-4653-48f8-894b-300fbae29db1 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:43.335289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-97adc5c9-4653-48f8-894b-300fbae29db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:43.431861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c235b-a455-4f8c-a04e-a17ddd666c6c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:43.454977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 125/238] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:43.479912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e311b5-07c8-4e5c-9fc1-8b270325a874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:43.480553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e311b5-07c8-4e5c-9fc1-8b270325a874 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:43.480828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e311b5-07c8-4e5c-9fc1-8b270325a874 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:43.481179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e311b5-07c8-4e5c-9fc1-8b270325a874 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:43.485345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:43.497357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e311b5-07c8-4e5c-9fc1-8b270325a874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-61e311b5-07c8-4e5c-9fc1-8b270325a874 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:43.502289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:43.503146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:43.503948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:43.506454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 114/239] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:43.509490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97adc5c9-4653-48f8-894b-300fbae29db1 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:43.513484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e311b5-07c8-4e5c-9fc1-8b270325a874 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:43.576322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:43.666157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e311b5-07c8-4e5c-9fc1-8b270325a874 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:43.704832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd43998-808c-4eea-9708-4904d5b28334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:43.705458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd43998-808c-4eea-9708-4904d5b28334 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:43.705737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd43998-808c-4eea-9708-4904d5b28334 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:43.706031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd43998-808c-4eea-9708-4904d5b28334 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:43.706693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd43998-808c-4eea-9708-4904d5b28334 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:43.706989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd43998-808c-4eea-9708-4904d5b28334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:43.709195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd43998-808c-4eea-9708-4904d5b28334 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:43.711423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97adc5c9-4653-48f8-894b-300fbae29db1 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:43.718787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 115/240] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:43.724722 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd43998-808c-4eea-9708-4904d5b28334 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:43.759208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cb910-3eeb-4469-b94d-14a47a9cfc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:43.760919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cb910-3eeb-4469-b94d-14a47a9cfc65 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:43.760919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cb910-3eeb-4469-b94d-14a47a9cfc65 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:43.760919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cb910-3eeb-4469-b94d-14a47a9cfc65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:43.761906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1cb910-3eeb-4469-b94d-14a47a9cfc65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:43.763452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1cb910-3eeb-4469-b94d-14a47a9cfc65 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:43.770228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 126/241] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:43.771255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cb910-3eeb-4469-b94d-14a47a9cfc65 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:43.841374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:43.904179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1cb910-3eeb-4469-b94d-14a47a9cfc65 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:43.914353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:43.921739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd43998-808c-4eea-9708-4904d5b28334 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:43.943094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3649df-8d3e-4c59-b6d0-e7b5b5cd7d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:43.944577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3649df-8d3e-4c59-b6d0-e7b5b5cd7d5f None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:43.944854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3649df-8d3e-4c59-b6d0-e7b5b5cd7d5f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:43.945117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3649df-8d3e-4c59-b6d0-e7b5b5cd7d5f None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:43.945765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3649df-8d3e-4c59-b6d0-e7b5b5cd7d5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:43.948031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3649df-8d3e-4c59-b6d0-e7b5b5cd7d5f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:43.990565 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a92583-2619-4552-945b-babd6826cd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:43.991457 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a92583-2619-4552-945b-babd6826cd4a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:43.992110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a92583-2619-4552-945b-babd6826cd4a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:43.992526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a92583-2619-4552-945b-babd6826cd4a None admin] PATH_INFO: `/v3/users/e2790d438a3a49629476be590cd5a46b` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.008924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a92583-2619-4552-945b-babd6826cd4a None admin] RBAC: Authorizing `identity:get_user(user_id=e2790d438a3a49629476be590cd5a46b)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:44.010862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a92583-2619-4552-945b-babd6826cd4a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:44.014848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 116/242] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:43 2022] GET /identity/v3/users/e2790d438a3a49629476be590cd5a46b => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:44.020722 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a92583-2619-4552-945b-babd6826cd4a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:44.061569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 127/243] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:31:43 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:44.140315 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a92583-2619-4552-945b-babd6826cd4a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:44.160263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:44.180788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba5ec9-d08a-4572-83e2-ad2bc061e94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:44.181499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba5ec9-d08a-4572-83e2-ad2bc061e94c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.181777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba5ec9-d08a-4572-83e2-ad2bc061e94c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.182128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba5ec9-d08a-4572-83e2-ad2bc061e94c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.236562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8297c-f94f-4e8d-9625-e17ba1a9a041 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.237077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8297c-f94f-4e8d-9625-e17ba1a9a041 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.237077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8297c-f94f-4e8d-9625-e17ba1a9a041 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.243261 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 128/244] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:44.244776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519c9e-564f-48d9-9a70-112e3be41095 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.245060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519c9e-564f-48d9-9a70-112e3be41095 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.245332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba519c9e-564f-48d9-9a70-112e3be41095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.268903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ba519c9e-564f-48d9-9a70-112e3be41095 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:44.276071 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ba519c9e-564f-48d9-9a70-112e3be41095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:44.281721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba5ec9-d08a-4572-83e2-ad2bc061e94c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:44.283292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba5ec9-d08a-4572-83e2-ad2bc061e94c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:44.390587 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 129/245] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:44.421284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba519c9e-564f-48d9-9a70-112e3be41095 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:44.429195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 117/246] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 16:31:44.437856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba5ec9-d08a-4572-83e2-ad2bc061e94c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:44.460002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:44.558748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:44.568321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba519c9e-564f-48d9-9a70-112e3be41095 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:44.581850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba5ec9-d08a-4572-83e2-ad2bc061e94c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:44.612727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d58f4b-c8a1-422f-9d4d-b05b3b28beb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:44.614274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d58f4b-c8a1-422f-9d4d-b05b3b28beb2 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.614692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d58f4b-c8a1-422f-9d4d-b05b3b28beb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.615124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d58f4b-c8a1-422f-9d4d-b05b3b28beb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.628119 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b981a-7c60-4446-8422-5bb4588e7378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:44.628548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d58f4b-c8a1-422f-9d4d-b05b3b28beb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:44.629674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b981a-7c60-4446-8422-5bb4588e7378 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.629674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b981a-7c60-4446-8422-5bb4588e7378 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.630102 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b981a-7c60-4446-8422-5bb4588e7378 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.631382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b981a-7c60-4446-8422-5bb4588e7378 None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:44.632854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b981a-7c60-4446-8422-5bb4588e7378 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:44.634777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 118/247] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:44 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:44.635898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06cce2-f1c2-4ff7-8af3-c9aa2a951e16 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-a5d58f4b-c8a1-422f-9d4d-b05b3b28beb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:44.636119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:44.637153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:44.638131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:44.638862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06cce2-f1c2-4ff7-8af3-c9aa2a951e16 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.638862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06cce2-f1c2-4ff7-8af3-c9aa2a951e16 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.639987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 119/248] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:44.640632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06cce2-f1c2-4ff7-8af3-c9aa2a951e16 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:44.641112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 130/249] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:31:44.648137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be31f4d-e887-42a2-9c1c-12b44344995e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.648456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be31f4d-e887-42a2-9c1c-12b44344995e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.653065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be31f4d-e887-42a2-9c1c-12b44344995e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.670177 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9be31f4d-e887-42a2-9c1c-12b44344995e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:44.671392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9be31f4d-e887-42a2-9c1c-12b44344995e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:44.760552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:44.761466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 131/250] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:44.762245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be31f4d-e887-42a2-9c1c-12b44344995e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:44.781142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06cce2-f1c2-4ff7-8af3-c9aa2a951e16 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:44.827279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582e6d1-e0d3-4c4f-873a-31e38f7477a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:44.827809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582e6d1-e0d3-4c4f-873a-31e38f7477a7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.828085 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582e6d1-e0d3-4c4f-873a-31e38f7477a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.828353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582e6d1-e0d3-4c4f-873a-31e38f7477a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.829084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582e6d1-e0d3-4c4f-873a-31e38f7477a7 None admin] RBAC: Adding query filter params (role.id=d7c03cc554a74d0aad88976354649d2d, scope.project.id=5796807ec0ad4cab845934a9df6b9689, user.id=e2790d438a3a49629476be590cd5a46b) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:44.829392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582e6d1-e0d3-4c4f-873a-31e38f7477a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:44.831417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582e6d1-e0d3-4c4f-873a-31e38f7477a7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:44.838524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 120/251] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:44 2022] GET /identity/v3/role_assignments?user.id=e2790d438a3a49629476be590cd5a46b&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=d7c03cc554a74d0aad88976354649d2d => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:44.839754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582e6d1-e0d3-4c4f-873a-31e38f7477a7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:44.870169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:44.890104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be31f4d-e887-42a2-9c1c-12b44344995e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:44.941338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c93df-def0-4713-b2b8-75da64453097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:44.942482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c93df-def0-4713-b2b8-75da64453097 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.942979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c93df-def0-4713-b2b8-75da64453097 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.943496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c93df-def0-4713-b2b8-75da64453097 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.944332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c93df-def0-4713-b2b8-75da64453097 None admin] RBAC: Adding query filter params (name=member) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:44.944815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c93df-def0-4713-b2b8-75da64453097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:44.946633 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c93df-def0-4713-b2b8-75da64453097 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:44.966492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 132/252] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:44.968119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b0e94-4590-4248-8031-667898b57b27 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.968615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b0e94-4590-4248-8031-667898b57b27 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.969043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b0e94-4590-4248-8031-667898b57b27 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.970625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 133/253] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:44.980185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1de9e2-733b-4c87-99ac-6dc3f9bedb99 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:44.980769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1de9e2-733b-4c87-99ac-6dc3f9bedb99 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:44.981211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1de9e2-733b-4c87-99ac-6dc3f9bedb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:44.999276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582e6d1-e0d3-4c4f-873a-31e38f7477a7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:45.010927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ed1de9e2-733b-4c87-99ac-6dc3f9bedb99 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:45.012521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ed1de9e2-733b-4c87-99ac-6dc3f9bedb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:45.064126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079f6f0-9699-4722-9686-ccf04b809064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:45.064828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f6f0-9699-4722-9686-ccf04b809064 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:45.065108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f6f0-9699-4722-9686-ccf04b809064 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:45.065376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079f6f0-9699-4722-9686-ccf04b809064 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:45.091893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079f6f0-9699-4722-9686-ccf04b809064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-6079f6f0-9699-4722-9686-ccf04b809064 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:45.099298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:45.100217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 121/254] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:31:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:45.101052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079f6f0-9699-4722-9686-ccf04b809064 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:45.116445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:45.138997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 134/255] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:45.140125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1de9e2-733b-4c87-99ac-6dc3f9bedb99 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:45.252388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:45.279365 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1de9e2-733b-4c87-99ac-6dc3f9bedb99 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:45.325527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85b059-d4db-457f-b85c-dd0b6f7880a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:45.327817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85b059-d4db-457f-b85c-dd0b6f7880a7 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:45.328249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85b059-d4db-457f-b85c-dd0b6f7880a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:45.328654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85b059-d4db-457f-b85c-dd0b6f7880a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:45.329612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079f6f0-9699-4722-9686-ccf04b809064 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:45.330791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85b059-d4db-457f-b85c-dd0b6f7880a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:45.331485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85b059-d4db-457f-b85c-dd0b6f7880a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:45.335119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85b059-d4db-457f-b85c-dd0b6f7880a7 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:45.353367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 135/256] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:45.357063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85b059-d4db-457f-b85c-dd0b6f7880a7 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:45.416436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db14eb-6292-4757-b8e3-bf6674f2aae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:45.417440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db14eb-6292-4757-b8e3-bf6674f2aae1 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:45.417855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db14eb-6292-4757-b8e3-bf6674f2aae1 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:45.418600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db14eb-6292-4757-b8e3-bf6674f2aae1 None admin] PATH_INFO: `/v3/users/6717eba732714a84bdfd052cf5aceb40` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:45.421606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db14eb-6292-4757-b8e3-bf6674f2aae1 None admin] RBAC: Authorizing `identity:get_user(user_id=6717eba732714a84bdfd052cf5aceb40)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:45.426985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db14eb-6292-4757-b8e3-bf6674f2aae1 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:45.431677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 122/257] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:45 2022] GET /identity/v3/users/6717eba732714a84bdfd052cf5aceb40 => generated 297 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:45.433362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db14eb-6292-4757-b8e3-bf6674f2aae1 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:45.542076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:45.566114 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db14eb-6292-4757-b8e3-bf6674f2aae1 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:45.603667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85b059-d4db-457f-b85c-dd0b6f7880a7 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:45.610006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9e4a2-9c9d-4868-b00d-d5e2aa406ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:45.610811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9e4a2-9c9d-4868-b00d-d5e2aa406ac4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:45.611225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9e4a2-9c9d-4868-b00d-d5e2aa406ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:45.611630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9e4a2-9c9d-4868-b00d-d5e2aa406ac4 None admin] PATH_INFO: `/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:45.614621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9e4a2-9c9d-4868-b00d-d5e2aa406ac4 None admin] RBAC: Authorizing `identity:get_user(user_id=d31d7aa5563f4253b885ce8ee7bbcb47)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:45.617998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9e4a2-9c9d-4868-b00d-d5e2aa406ac4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:45.617998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 123/258] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:45 2022] GET /identity/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:45.619678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9e4a2-9c9d-4868-b00d-d5e2aa406ac4 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:45.652002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da9c3a-27d9-491e-9a34-dec85ac06dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:45.652925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da9c3a-27d9-491e-9a34-dec85ac06dd3 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:45.653369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da9c3a-27d9-491e-9a34-dec85ac06dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:45.653772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da9c3a-27d9-491e-9a34-dec85ac06dd3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:45.664280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da9c3a-27d9-491e-9a34-dec85ac06dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-52da9c3a-27d9-491e-9a34-dec85ac06dd3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:45.669680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:45.670762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:45.671723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:45.674024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 136/259] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:31:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:45.675039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da9c3a-27d9-491e-9a34-dec85ac06dd3 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:45.701559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:45.857620 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9e4a2-9c9d-4868-b00d-d5e2aa406ac4 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:45.871156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da9c3a-27d9-491e-9a34-dec85ac06dd3 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:45.895829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4f4c9-530e-4a97-a706-4c0903c48d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:45.898556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4f4c9-530e-4a97-a706-4c0903c48d6c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:45.898837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4f4c9-530e-4a97-a706-4c0903c48d6c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:45.899110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4f4c9-530e-4a97-a706-4c0903c48d6c None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:45.900915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4f4c9-530e-4a97-a706-4c0903c48d6c None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:45.902492 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4f4c9-530e-4a97-a706-4c0903c48d6c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:45.904146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 124/260] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:45 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:45.910363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4f4c9-530e-4a97-a706-4c0903c48d6c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:45.944372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:45.946749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49e9f6-2a26-4bda-bec3-759a8ce5fcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:45.948795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49e9f6-2a26-4bda-bec3-759a8ce5fcfb None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:45.949077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49e9f6-2a26-4bda-bec3-759a8ce5fcfb None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:45.949348 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad49e9f6-2a26-4bda-bec3-759a8ce5fcfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:46.070363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:46.076432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49e9f6-2a26-4bda-bec3-759a8ce5fcfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:46.079574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad49e9f6-2a26-4bda-bec3-759a8ce5fcfb None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:46.113435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4f4c9-530e-4a97-a706-4c0903c48d6c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:46.188290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dd1e4-6bb4-40d5-ac55-f7f2fad52b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:46.188897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dd1e4-6bb4-40d5-ac55-f7f2fad52b22 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:46.189173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dd1e4-6bb4-40d5-ac55-f7f2fad52b22 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:46.191297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dd1e4-6bb4-40d5-ac55-f7f2fad52b22 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:46.192065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960dd1e4-6bb4-40d5-ac55-f7f2fad52b22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:46.192368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960dd1e4-6bb4-40d5-ac55-f7f2fad52b22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:46.193726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960dd1e4-6bb4-40d5-ac55-f7f2fad52b22 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:46.214546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 125/261] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:31:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:46.216639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dd1e4-6bb4-40d5-ac55-f7f2fad52b22 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:46.228653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 137/262] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:46.229525 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49e9f6-2a26-4bda-bec3-759a8ce5fcfb None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:46.353538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:46.386629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:46.392574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad49e9f6-2a26-4bda-bec3-759a8ce5fcfb None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:46.402511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dd1e4-6bb4-40d5-ac55-f7f2fad52b22 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:46.456763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661103f-3506-4720-a3e9-92eaea30e064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:46.457388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661103f-3506-4720-a3e9-92eaea30e064 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:46.458249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661103f-3506-4720-a3e9-92eaea30e064 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:46.458522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661103f-3506-4720-a3e9-92eaea30e064 None admin] PATH_INFO: `/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:46.461513 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812fdd3-63fe-4a8f-90c6-2609c598f5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:46.462270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812fdd3-63fe-4a8f-90c6-2609c598f5a0 None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:46.462544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812fdd3-63fe-4a8f-90c6-2609c598f5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:46.462812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812fdd3-63fe-4a8f-90c6-2609c598f5a0 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689/users/6717eba732714a84bdfd052cf5aceb40/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:46.464479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661103f-3506-4720-a3e9-92eaea30e064 None admin] RBAC: Authorizing `identity:get_user(user_id=d31d7aa5563f4253b885ce8ee7bbcb47)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:46.466132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661103f-3506-4720-a3e9-92eaea30e064 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:46.469040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812fdd3-63fe-4a8f-90c6-2609c598f5a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5796807ec0ad4cab845934a9df6b9689, user_id=6717eba732714a84bdfd052cf5aceb40, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:46.470504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812fdd3-63fe-4a8f-90c6-2609c598f5a0 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:46.472129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 138/263] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:46 2022] GET /identity/v3/users/d31d7aa5563f4253b885ce8ee7bbcb47 => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:46.472997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661103f-3506-4720-a3e9-92eaea30e064 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:46.559405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 126/264] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:31:46 2022] PUT /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689/users/6717eba732714a84bdfd052cf5aceb40/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:31:46.560251 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812fdd3-63fe-4a8f-90c6-2609c598f5a0 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:46.653034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661103f-3506-4720-a3e9-92eaea30e064 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:46.724143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c5d94-2a48-408f-b1b2-ce949b0bde94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:46.725672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c5d94-2a48-408f-b1b2-ce949b0bde94 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:46.725946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c5d94-2a48-408f-b1b2-ce949b0bde94 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:46.726236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c5d94-2a48-408f-b1b2-ce949b0bde94 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:46.727648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c5d94-2a48-408f-b1b2-ce949b0bde94 None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:46.729107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c5d94-2a48-408f-b1b2-ce949b0bde94 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:46.740513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 139/265] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:46 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:46.746571 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c5d94-2a48-408f-b1b2-ce949b0bde94 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:46.762047 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812fdd3-63fe-4a8f-90c6-2609c598f5a0 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:46.801390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:46.827243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4ec69-62d6-4a85-898d-c575b576d846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:46.827861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4ec69-62d6-4a85-898d-c575b576d846 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:46.828136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4ec69-62d6-4a85-898d-c575b576d846 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:46.828637 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4ec69-62d6-4a85-898d-c575b576d846 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:46.835727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:46.924338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c5d94-2a48-408f-b1b2-ce949b0bde94 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:46.956935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4ec69-62d6-4a85-898d-c575b576d846 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:46.959425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b4ec69-62d6-4a85-898d-c575b576d846 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:47.010743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c2f8a-dbca-43af-b518-429a80d9fc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:47.011988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2f8a-dbca-43af-b518-429a80d9fc31 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.012487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2f8a-dbca-43af-b518-429a80d9fc31 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.012978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2f8a-dbca-43af-b518-429a80d9fc31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.013984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c2f8a-dbca-43af-b518-429a80d9fc31 None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=5796807ec0ad4cab845934a9df6b9689, user.id=d31d7aa5563f4253b885ce8ee7bbcb47) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:47.014508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c2f8a-dbca-43af-b518-429a80d9fc31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:47.016186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c2f8a-dbca-43af-b518-429a80d9fc31 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:47.036581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 140/266] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:46 2022] GET /identity/v3/role_assignments?user.id=d31d7aa5563f4253b885ce8ee7bbcb47&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:47.038245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab713e-9a48-48bf-b8d7-a600fd3e3c04 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.038772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab713e-9a48-48bf-b8d7-a600fd3e3c04 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.039244 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab713e-9a48-48bf-b8d7-a600fd3e3c04 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.050417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 141/267] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:47 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:47.052169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78d01d-b6fc-46f7-956c-b4f19575486d None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.052696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78d01d-b6fc-46f7-956c-b4f19575486d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.053239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78d01d-b6fc-46f7-956c-b4f19575486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.081201 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 127/268] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:47.082667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd13374-0ec2-4522-84b4-9d3f29472b81 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.083473 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd13374-0ec2-4522-84b4-9d3f29472b81 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.083744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd13374-0ec2-4522-84b4-9d3f29472b81 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.085281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 128/269] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:47.087182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-2d78d01d-b6fc-46f7-956c-b4f19575486d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:47.088673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2d78d01d-b6fc-46f7-956c-b4f19575486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:47.090946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd13374-0ec2-4522-84b4-9d3f29472b81 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:47.199333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 142/270] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:47.201005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:47.201727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f82b-44c8-453c-9ceb-104c8f014300 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.202230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f82b-44c8-453c-9ceb-104c8f014300 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.202662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f82b-44c8-453c-9ceb-104c8f014300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.224047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ef46f82b-44c8-453c-9ceb-104c8f014300 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:47.225525 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ef46f82b-44c8-453c-9ceb-104c8f014300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:47.251887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:47.299062 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 143/271] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:47.300138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46f82b-44c8-453c-9ceb-104c8f014300 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:47.322778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd13374-0ec2-4522-84b4-9d3f29472b81 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:47.384833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270832f3-64fa-4df1-8006-72e052fb4ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:47.385492 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270832f3-64fa-4df1-8006-72e052fb4ab6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.385735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270832f3-64fa-4df1-8006-72e052fb4ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.386206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270832f3-64fa-4df1-8006-72e052fb4ab6 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.388614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270832f3-64fa-4df1-8006-72e052fb4ab6 None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:47.389657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270832f3-64fa-4df1-8006-72e052fb4ab6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:47.392727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 129/272] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:47 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:47.393514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270832f3-64fa-4df1-8006-72e052fb4ab6 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:47.438102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46f82b-44c8-453c-9ceb-104c8f014300 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:47.478278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5d07a-08de-47f6-b0f5-1d4e3f9b5be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:47.479391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d07a-08de-47f6-b0f5-1d4e3f9b5be1 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.479884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d07a-08de-47f6-b0f5-1d4e3f9b5be1 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.480664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5d07a-08de-47f6-b0f5-1d4e3f9b5be1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.495239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5d07a-08de-47f6-b0f5-1d4e3f9b5be1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-22e5d07a-08de-47f6-b0f5-1d4e3f9b5be1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:47.501302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:47.502231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:47.503006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:47.506219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 144/273] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:47.510102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5d07a-08de-47f6-b0f5-1d4e3f9b5be1 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:47.534218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:47.538130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270832f3-64fa-4df1-8006-72e052fb4ab6 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:47.539701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:47.574375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea808391-e480-4ba5-ab16-d7b0edc34eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:47.575009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808391-e480-4ba5-ab16-d7b0edc34eef None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.575282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808391-e480-4ba5-ab16-d7b0edc34eef None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.575551 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea808391-e480-4ba5-ab16-d7b0edc34eef None admin] PATH_INFO: `/v3/domains/default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.576678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea808391-e480-4ba5-ab16-d7b0edc34eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:47.578165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea808391-e480-4ba5-ab16-d7b0edc34eef None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:47.580017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 130/274] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:47.580661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea808391-e480-4ba5-ab16-d7b0edc34eef None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:47.628552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5d07a-08de-47f6-b0f5-1d4e3f9b5be1 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:47.667143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32755f51-78e6-411e-8182-3a0efe550b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:47.668078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32755f51-78e6-411e-8182-3a0efe550b32 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.668355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32755f51-78e6-411e-8182-3a0efe550b32 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.668616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32755f51-78e6-411e-8182-3a0efe550b32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.670026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32755f51-78e6-411e-8182-3a0efe550b32 None admin] RBAC: Adding query filter params (role.id=20d35b10934447938237c0db5202bbfb, scope.project.id=5796807ec0ad4cab845934a9df6b9689, user.id=d31d7aa5563f4253b885ce8ee7bbcb47) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:47.670395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32755f51-78e6-411e-8182-3a0efe550b32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:47.671858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32755f51-78e6-411e-8182-3a0efe550b32 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:47.698594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 145/275] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:47 2022] GET /identity/v3/role_assignments?user.id=d31d7aa5563f4253b885ce8ee7bbcb47&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=20d35b10934447938237c0db5202bbfb => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:47.699087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea808391-e480-4ba5-ab16-d7b0edc34eef None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:47.700341 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41a0cd-1643-4758-9521-68f9af3d4538 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.700814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41a0cd-1643-4758-9521-68f9af3d4538 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.701084 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41a0cd-1643-4758-9521-68f9af3d4538 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.702817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 146/276] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:47.708169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41a0cd-1643-4758-9521-68f9af3d4538 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:47.749917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485a367-7934-4ef0-a5b1-dd8592dd4c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:47.750942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485a367-7934-4ef0-a5b1-dd8592dd4c55 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.751439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485a367-7934-4ef0-a5b1-dd8592dd4c55 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.751899 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485a367-7934-4ef0-a5b1-dd8592dd4c55 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.752779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485a367-7934-4ef0-a5b1-dd8592dd4c55 None admin] RBAC: Adding query filter params (name=member) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:47.753277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485a367-7934-4ef0-a5b1-dd8592dd4c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:47.755708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485a367-7934-4ef0-a5b1-dd8592dd4c55 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:47.768602 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 131/277] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:47.770440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f77ed-cd5a-4181-aef8-cdd1d2e831bd None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.770968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f77ed-cd5a-4181-aef8-cdd1d2e831bd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.771453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f77ed-cd5a-4181-aef8-cdd1d2e831bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.797480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-909f77ed-cd5a-4181-aef8-cdd1d2e831bd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:47.798850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-909f77ed-cd5a-4181-aef8-cdd1d2e831bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:47.816745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:47.831012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:47.841926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41a0cd-1643-4758-9521-68f9af3d4538 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:47.886855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407eb754-7e75-4995-a9c2-67275e991220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:47.887461 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eb754-7e75-4995-a9c2-67275e991220 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:47.887741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eb754-7e75-4995-a9c2-67275e991220 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:47.888008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eb754-7e75-4995-a9c2-67275e991220 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:47.888677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407eb754-7e75-4995-a9c2-67275e991220 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:47.890182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407eb754-7e75-4995-a9c2-67275e991220 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:47.896994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 132/278] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:47.898172 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f77ed-cd5a-4181-aef8-cdd1d2e831bd None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:47.992620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 147/279] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:31:47 2022] POST /identity/v3/users => generated 293 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:47.993972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407eb754-7e75-4995-a9c2-67275e991220 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:48.037421 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f77ed-cd5a-4181-aef8-cdd1d2e831bd None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:48.079798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac60ad-11f0-4827-8b22-f615698038bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:48.080444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac60ad-11f0-4827-8b22-f615698038bd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:48.080725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac60ad-11f0-4827-8b22-f615698038bd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:48.080995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac60ad-11f0-4827-8b22-f615698038bd None admin] PATH_INFO: `/v3/users/add7a433ab424e4b9152da210a551eed` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:48.097472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac60ad-11f0-4827-8b22-f615698038bd None admin] RBAC: Authorizing `identity:get_user(user_id=add7a433ab424e4b9152da210a551eed)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:48.099564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac60ad-11f0-4827-8b22-f615698038bd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:48.101359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 133/280] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:47 2022] GET /identity/v3/users/add7a433ab424e4b9152da210a551eed => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:48.107255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac60ad-11f0-4827-8b22-f615698038bd None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:48.130583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407eb754-7e75-4995-a9c2-67275e991220 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:48.131305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:48.171432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda7d70-67c9-46f3-804e-08e3b1ad11c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:48.172058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7d70-67c9-46f3-804e-08e3b1ad11c4 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:48.172333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7d70-67c9-46f3-804e-08e3b1ad11c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:48.172600 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda7d70-67c9-46f3-804e-08e3b1ad11c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:48.180603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:48.189259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda7d70-67c9-46f3-804e-08e3b1ad11c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-ccda7d70-67c9-46f3-804e-08e3b1ad11c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:48.195985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:48.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:48.197718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 148/281] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:48.204609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda7d70-67c9-46f3-804e-08e3b1ad11c4 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:48.239739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac60ad-11f0-4827-8b22-f615698038bd None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:48.284450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a0adc-6205-4b59-afe1-c80488ee184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:48.285382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0adc-6205-4b59-afe1-c80488ee184d None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:48.285604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0adc-6205-4b59-afe1-c80488ee184d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:48.286638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a0adc-6205-4b59-afe1-c80488ee184d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:48.287392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a0adc-6205-4b59-afe1-c80488ee184d None admin] RBAC: Adding query filter params (role.id=2490390ea61f4cf594df3587e7674650, user.id=add7a433ab424e4b9152da210a551eed, scope.system=all) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:48.287752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a0adc-6205-4b59-afe1-c80488ee184d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:48.289367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a0adc-6205-4b59-afe1-c80488ee184d None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:48.298845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 134/282] 10.222.0.79 () {56 vars in 1308 bytes} [Tue Aug 9 16:31:48 2022] GET /identity/v3/role_assignments?user.id=add7a433ab424e4b9152da210a551eed&scope.system=all&role.id=2490390ea61f4cf594df3587e7674650 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:48.332026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda7d70-67c9-46f3-804e-08e3b1ad11c4 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:48.373350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8aec-fff0-4e0b-9c7c-3fa9466570bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:48.374261 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8aec-fff0-4e0b-9c7c-3fa9466570bc None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:48.374673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8aec-fff0-4e0b-9c7c-3fa9466570bc None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:48.375075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e8aec-fff0-4e0b-9c7c-3fa9466570bc None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:48.375906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8aec-fff0-4e0b-9c7c-3fa9466570bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:48.376354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8aec-fff0-4e0b-9c7c-3fa9466570bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:48.377973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e8aec-fff0-4e0b-9c7c-3fa9466570bc None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:48.388068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 149/283] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:48.394050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a0adc-6205-4b59-afe1-c80488ee184d None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:48.500782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:48.517289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a0adc-6205-4b59-afe1-c80488ee184d None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:48.555243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbecdb-bb2b-4e48-83f1-4e61b9236762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:48.555793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbecdb-bb2b-4e48-83f1-4e61b9236762 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:48.556065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbecdb-bb2b-4e48-83f1-4e61b9236762 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:48.556330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbecdb-bb2b-4e48-83f1-4e61b9236762 None admin] PATH_INFO: `/v3/users/e2790d438a3a49629476be590cd5a46b` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:48.557456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbecdb-bb2b-4e48-83f1-4e61b9236762 None admin] RBAC: Authorizing `identity:get_user(user_id=e2790d438a3a49629476be590cd5a46b)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:48.558998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbecdb-bb2b-4e48-83f1-4e61b9236762 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:48.560857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 135/284] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:48 2022] GET /identity/v3/users/e2790d438a3a49629476be590cd5a46b => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:48.566554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8aec-fff0-4e0b-9c7c-3fa9466570bc None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:48.655075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:48.716813 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e8aec-fff0-4e0b-9c7c-3fa9466570bc None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:48.753336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8612a725-a22c-4144-b789-334ae1d79675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:48.754372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612a725-a22c-4144-b789-334ae1d79675 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:48.754881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612a725-a22c-4144-b789-334ae1d79675 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:48.755346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8612a725-a22c-4144-b789-334ae1d79675 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:48.756716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612a725-a22c-4144-b789-334ae1d79675 None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:48.758424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8612a725-a22c-4144-b789-334ae1d79675 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:48.760673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 150/285] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:48 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:48.766125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbecdb-bb2b-4e48-83f1-4e61b9236762 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:48.913438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbecdb-bb2b-4e48-83f1-4e61b9236762 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:48.969481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbe1b2-172b-4e6e-954d-0b11eb022fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:48.970193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbe1b2-172b-4e6e-954d-0b11eb022fd1 None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:48.970475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbe1b2-172b-4e6e-954d-0b11eb022fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:48.970745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbe1b2-172b-4e6e-954d-0b11eb022fd1 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689/users/e2790d438a3a49629476be590cd5a46b/roles/d7c03cc554a74d0aad88976354649d2d` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:48.975231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbe1b2-172b-4e6e-954d-0b11eb022fd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5796807ec0ad4cab845934a9df6b9689, user_id=e2790d438a3a49629476be590cd5a46b, role_id=d7c03cc554a74d0aad88976354649d2d)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:48.976683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbe1b2-172b-4e6e-954d-0b11eb022fd1 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:49.025140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afee9f-4444-4536-b69f-2ba46b861eef None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:49.025463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afee9f-4444-4536-b69f-2ba46b861eef None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:49.025687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afee9f-4444-4536-b69f-2ba46b861eef None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:49.027400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 151/286] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:49.032904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb633f-3a09-4be3-85a7-25d59e40b62d None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:49.033207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb633f-3a09-4be3-85a7-25d59e40b62d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:49.033443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb633f-3a09-4be3-85a7-25d59e40b62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:49.052564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-04cb633f-3a09-4be3-85a7-25d59e40b62d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:49.053744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-04cb633f-3a09-4be3-85a7-25d59e40b62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:49.068844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 136/287] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:31:48 2022] PUT /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689/users/e2790d438a3a49629476be590cd5a46b/roles/d7c03cc554a74d0aad88976354649d2d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:31:49.127150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 152/288] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:49.150561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbe1b2-172b-4e6e-954d-0b11eb022fd1 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:49.172048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:49.324354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbe1b2-172b-4e6e-954d-0b11eb022fd1 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:49.381741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe351d-73a0-449b-b723-68753583204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:49.382583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe351d-73a0-449b-b723-68753583204f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:49.383002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe351d-73a0-449b-b723-68753583204f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:49.383373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe351d-73a0-449b-b723-68753583204f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:49.398350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe351d-73a0-449b-b723-68753583204f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-4cfe351d-73a0-449b-b723-68753583204f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:49.404456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:49.405341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:49.406339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 137/289] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:49.411552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb633f-3a09-4be3-85a7-25d59e40b62d None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:49.510877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:49.547190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb633f-3a09-4be3-85a7-25d59e40b62d None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:49.584581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37cc56-a0e7-4c14-bed4-8327f12a973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:49.585200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37cc56-a0e7-4c14-bed4-8327f12a973f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:49.585447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37cc56-a0e7-4c14-bed4-8327f12a973f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:49.585705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37cc56-a0e7-4c14-bed4-8327f12a973f None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:49.586406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37cc56-a0e7-4c14-bed4-8327f12a973f None admin] RBAC: Adding query filter params (name=member) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:49.586701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37cc56-a0e7-4c14-bed4-8327f12a973f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:49.588078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37cc56-a0e7-4c14-bed4-8327f12a973f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:49.597604 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 153/290] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:49.603123 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe351d-73a0-449b-b723-68753583204f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:49.726394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:49.765770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe351d-73a0-449b-b723-68753583204f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:49.867798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b1ba1-0da5-475d-8027-7c9e01e9ecfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:49.868605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b1ba1-0da5-475d-8027-7c9e01e9ecfb None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:49.868952 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b1ba1-0da5-475d-8027-7c9e01e9ecfb None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:49.869334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b1ba1-0da5-475d-8027-7c9e01e9ecfb None admin] PATH_INFO: `/v3/users/6717eba732714a84bdfd052cf5aceb40` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:49.871027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b1ba1-0da5-475d-8027-7c9e01e9ecfb None admin] RBAC: Authorizing `identity:get_user(user_id=6717eba732714a84bdfd052cf5aceb40)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:49.873065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b1ba1-0da5-475d-8027-7c9e01e9ecfb None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:49.875708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 138/291] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:49 2022] GET /identity/v3/users/6717eba732714a84bdfd052cf5aceb40 => generated 297 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:49.882773 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37cc56-a0e7-4c14-bed4-8327f12a973f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:49.994507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37cc56-a0e7-4c14-bed4-8327f12a973f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:50.015246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:50.033086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42cf98-8a49-406b-a7b1-3ce9814a28a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:50.033086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42cf98-8a49-406b-a7b1-3ce9814a28a9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:50.033086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42cf98-8a49-406b-a7b1-3ce9814a28a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:50.033086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42cf98-8a49-406b-a7b1-3ce9814a28a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:50.099632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42cf98-8a49-406b-a7b1-3ce9814a28a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:50.101089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42cf98-8a49-406b-a7b1-3ce9814a28a9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:50.204300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 154/292] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:50.210064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b1ba1-0da5-475d-8027-7c9e01e9ecfb None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:50.373665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b1ba1-0da5-475d-8027-7c9e01e9ecfb None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:50.427628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca56aa-d354-4d9c-8afd-3c3bc67ec986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:50.428298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca56aa-d354-4d9c-8afd-3c3bc67ec986 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:50.428510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca56aa-d354-4d9c-8afd-3c3bc67ec986 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:50.428773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca56aa-d354-4d9c-8afd-3c3bc67ec986 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:50.429922 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca56aa-d354-4d9c-8afd-3c3bc67ec986 None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:50.431363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca56aa-d354-4d9c-8afd-3c3bc67ec986 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:50.433279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 139/293] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:50 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:50.439169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42cf98-8a49-406b-a7b1-3ce9814a28a9 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:50.478745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:50.580468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42cf98-8a49-406b-a7b1-3ce9814a28a9 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:50.621489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e2907-ef4f-45ba-9666-3f8b9f9b94a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:50.622423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e2907-ef4f-45ba-9666-3f8b9f9b94a9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:50.622635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e2907-ef4f-45ba-9666-3f8b9f9b94a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:50.622788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e2907-ef4f-45ba-9666-3f8b9f9b94a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:50.623573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e2907-ef4f-45ba-9666-3f8b9f9b94a9 None admin] RBAC: Adding query filter params (role.id=2490390ea61f4cf594df3587e7674650, scope.project.id=5796807ec0ad4cab845934a9df6b9689, user.id=6717eba732714a84bdfd052cf5aceb40) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:50.623949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e2907-ef4f-45ba-9666-3f8b9f9b94a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:50.625315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e2907-ef4f-45ba-9666-3f8b9f9b94a9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:50.632204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 155/294] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:50 2022] GET /identity/v3/role_assignments?user.id=6717eba732714a84bdfd052cf5aceb40&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=2490390ea61f4cf594df3587e7674650 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:50.679395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88581eaa-c769-42b6-9fe3-654736270f2d None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:50.679791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88581eaa-c769-42b6-9fe3-654736270f2d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:50.680062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88581eaa-c769-42b6-9fe3-654736270f2d None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:50.681436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 140/295] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:50.687312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ad277-8309-40df-a98d-979de15e05e4 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:50.687608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ad277-8309-40df-a98d-979de15e05e4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:50.687855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ad277-8309-40df-a98d-979de15e05e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:50.707404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ca8ad277-8309-40df-a98d-979de15e05e4 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:50.708809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ca8ad277-8309-40df-a98d-979de15e05e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:50.787566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 156/296] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:50.810651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88581eaa-c769-42b6-9fe3-654736270f2d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:50.822157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:50.935765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88581eaa-c769-42b6-9fe3-654736270f2d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:50.974562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511e62e-1ff9-4fcd-8ea1-72421dc5d8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:50.975193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e62e-1ff9-4fcd-8ea1-72421dc5d8d3 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:50.975471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e62e-1ff9-4fcd-8ea1-72421dc5d8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:50.975743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e62e-1ff9-4fcd-8ea1-72421dc5d8d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:50.976905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511e62e-1ff9-4fcd-8ea1-72421dc5d8d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:50.978363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6511e62e-1ff9-4fcd-8ea1-72421dc5d8d3 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:50.980309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 141/297] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:50.986667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ad277-8309-40df-a98d-979de15e05e4 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:51.002438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:51.009771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a0e3e-103e-4d30-aa15-a96bbf39a4f6 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.010109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a0e3e-103e-4d30-aa15-a96bbf39a4f6 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.010391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a0e3e-103e-4d30-aa15-a96bbf39a4f6 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.012068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 142/298] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:51.024465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2035c-d8df-4f29-9dd4-479646580e75 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.024738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2035c-d8df-4f29-9dd4-479646580e75 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.025033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2035c-d8df-4f29-9dd4-479646580e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.051372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-e4d2035c-d8df-4f29-9dd4-479646580e75 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:51.053014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e4d2035c-d8df-4f29-9dd4-479646580e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:51.135801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ad277-8309-40df-a98d-979de15e05e4 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:51.151616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 143/299] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:51.155192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92161a-7f41-4517-a107-cb7f6ebace75 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.155192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92161a-7f41-4517-a107-cb7f6ebace75 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.155192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92161a-7f41-4517-a107-cb7f6ebace75 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.155192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 144/300] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:51.160205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd78e3-1adc-4d23-8a1f-ae55a71c2a41 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.160484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd78e3-1adc-4d23-8a1f-ae55a71c2a41 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.160752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd78e3-1adc-4d23-8a1f-ae55a71c2a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.175214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:51.182316 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c7dd78e3-1adc-4d23-8a1f-ae55a71c2a41 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:51.183542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c7dd78e3-1adc-4d23-8a1f-ae55a71c2a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:51.184204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96823b-7e86-4ff2-8781-5ebb42e02ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:51.185023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96823b-7e86-4ff2-8781-5ebb42e02ee1 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.185495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96823b-7e86-4ff2-8781-5ebb42e02ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.185750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96823b-7e86-4ff2-8781-5ebb42e02ee1 None admin] PATH_INFO: `/v3/groups` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.189398 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96823b-7e86-4ff2-8781-5ebb42e02ee1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:51.189398 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96823b-7e86-4ff2-8781-5ebb42e02ee1 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:51.273979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 157/301] 10.222.0.79 () {60 vars in 1134 bytes} [Tue Aug 9 16:31:50 2022] POST /identity/v3/groups => generated 229 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:51.273979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96823b-7e86-4ff2-8781-5ebb42e02ee1 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:51.276977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 145/302] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:31:51.305142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd78e3-1adc-4d23-8a1f-ae55a71c2a41 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:51.405985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:51.414258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96823b-7e86-4ff2-8781-5ebb42e02ee1 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:51.456643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd78e3-1adc-4d23-8a1f-ae55a71c2a41 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:51.467523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da976ef8-6341-4fe7-acca-dc3d98922d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:51.468425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da976ef8-6341-4fe7-acca-dc3d98922d67 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.468845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da976ef8-6341-4fe7-acca-dc3d98922d67 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.469247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da976ef8-6341-4fe7-acca-dc3d98922d67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.482204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da976ef8-6341-4fe7-acca-dc3d98922d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-da976ef8-6341-4fe7-acca-dc3d98922d67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:51.487746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:51.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:51.489489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 158/303] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:51.496477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da976ef8-6341-4fe7-acca-dc3d98922d67 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:51.510175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3cf62-f835-408d-8dc3-a8607aa1218f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:51.510815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3cf62-f835-408d-8dc3-a8607aa1218f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.511116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3cf62-f835-408d-8dc3-a8607aa1218f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.511642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3cf62-f835-408d-8dc3-a8607aa1218f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.515567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:51.528628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3cf62-f835-408d-8dc3-a8607aa1218f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-f6f3cf62-f835-408d-8dc3-a8607aa1218f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:51.534036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:51.535009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:51.536261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 146/304] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:51.542207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3cf62-f835-408d-8dc3-a8607aa1218f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:51.628210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da976ef8-6341-4fe7-acca-dc3d98922d67 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:51.675499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34795e0b-ba0d-4b31-8dd2-15ed68e84d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:51.677113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795e0b-ba0d-4b31-8dd2-15ed68e84d15 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.677514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795e0b-ba0d-4b31-8dd2-15ed68e84d15 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.677876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795e0b-ba0d-4b31-8dd2-15ed68e84d15 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.678676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34795e0b-ba0d-4b31-8dd2-15ed68e84d15 None admin] RBAC: Adding query filter params (name=member) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:51.679073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34795e0b-ba0d-4b31-8dd2-15ed68e84d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:51.680620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34795e0b-ba0d-4b31-8dd2-15ed68e84d15 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:51.682429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:51.689133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3cf62-f835-408d-8dc3-a8607aa1218f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:51.690816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 159/305] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:51.697476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34795e0b-ba0d-4b31-8dd2-15ed68e84d15 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:51.732381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9d98d-0e40-449d-8a8f-9d1bd1f1d9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:51.733240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9d98d-0e40-449d-8a8f-9d1bd1f1d9df None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.733770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9d98d-0e40-449d-8a8f-9d1bd1f1d9df None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.734213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9d98d-0e40-449d-8a8f-9d1bd1f1d9df None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.735619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9d98d-0e40-449d-8a8f-9d1bd1f1d9df None admin] RBAC: Adding query filter params (name=reader) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:51.736063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9d98d-0e40-449d-8a8f-9d1bd1f1d9df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:51.738147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9d98d-0e40-449d-8a8f-9d1bd1f1d9df None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:51.752191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 147/306] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:51.762087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9d98d-0e40-449d-8a8f-9d1bd1f1d9df None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:51.832311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34795e0b-ba0d-4b31-8dd2-15ed68e84d15 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:51.838887 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:51.869408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c699b-b5bc-4f8b-b101-912cb0c24412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:51.870391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c699b-b5bc-4f8b-b101-912cb0c24412 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.870843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c699b-b5bc-4f8b-b101-912cb0c24412 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.871294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c699b-b5bc-4f8b-b101-912cb0c24412 None admin] PATH_INFO: `/v3/users/add7a433ab424e4b9152da210a551eed` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.872429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c699b-b5bc-4f8b-b101-912cb0c24412 None admin] RBAC: Authorizing `identity:get_user(user_id=add7a433ab424e4b9152da210a551eed)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:51.873919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6c699b-b5bc-4f8b-b101-912cb0c24412 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:51.875583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 160/307] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:51 2022] GET /identity/v3/users/add7a433ab424e4b9152da210a551eed => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:51.876712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba2512-4b75-4452-b235-b8440179d9cd None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.876995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba2512-4b75-4452-b235-b8440179d9cd None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.877243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba2512-4b75-4452-b235-b8440179d9cd None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.878599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 161/308] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:51.883729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba2512-4b75-4452-b235-b8440179d9cd None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:51.890503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9d98d-0e40-449d-8a8f-9d1bd1f1d9df None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:51.929213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c1058-8881-49e4-bc4f-b0e54b4c01f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:51.929831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1058-8881-49e4-bc4f-b0e54b4c01f4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.930132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1058-8881-49e4-bc4f-b0e54b4c01f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.930401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c1058-8881-49e4-bc4f-b0e54b4c01f4 None admin] PATH_INFO: `/v3/users/32d928ad18fd42edbd4e2e88be910eaf` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.944983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c1058-8881-49e4-bc4f-b0e54b4c01f4 None admin] RBAC: Authorizing `identity:get_user(user_id=32d928ad18fd42edbd4e2e88be910eaf)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:51.946454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c1058-8881-49e4-bc4f-b0e54b4c01f4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:51.952660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 148/309] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:51 2022] GET /identity/v3/users/32d928ad18fd42edbd4e2e88be910eaf => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:51.952660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a4116-6cbf-4b16-a010-74340f9a0152 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:51.952660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a4116-6cbf-4b16-a010-74340f9a0152 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:51.952660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a4116-6cbf-4b16-a010-74340f9a0152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:51.960081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:51.968250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9f8a4116-6cbf-4b16-a010-74340f9a0152 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:51.969431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9f8a4116-6cbf-4b16-a010-74340f9a0152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:52.003306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba2512-4b75-4452-b235-b8440179d9cd None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:52.042238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 149/310] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:52.043397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a4116-6cbf-4b16-a010-74340f9a0152 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:52.050437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71ba6-a185-4cf8-8219-86e859ce0e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:52.051374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71ba6-a185-4cf8-8219-86e859ce0e6c None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:52.051945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71ba6-a185-4cf8-8219-86e859ce0e6c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:52.053509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71ba6-a185-4cf8-8219-86e859ce0e6c None admin] PATH_INFO: `/v3/system/users/add7a433ab424e4b9152da210a551eed/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:52.055785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab71ba6-a185-4cf8-8219-86e859ce0e6c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=add7a433ab424e4b9152da210a551eed, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:52.057460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab71ba6-a185-4cf8-8219-86e859ce0e6c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:52.097099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:52.141985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 162/311] 10.222.0.79 () {58 vars in 1334 bytes} [Tue Aug 9 16:31:51 2022] PUT /identity/v3/system/users/add7a433ab424e4b9152da210a551eed/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:31:52.141985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71ba6-a185-4cf8-8219-86e859ce0e6c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:52.167458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a4116-6cbf-4b16-a010-74340f9a0152 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:52.216134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83e6a1-13ca-45b9-be05-2ac401f22b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:52.216747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83e6a1-13ca-45b9-be05-2ac401f22b56 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:52.217015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83e6a1-13ca-45b9-be05-2ac401f22b56 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:52.217274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83e6a1-13ca-45b9-be05-2ac401f22b56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:52.218043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83e6a1-13ca-45b9-be05-2ac401f22b56 None admin] RBAC: Adding query filter params (role.id=d7c03cc554a74d0aad88976354649d2d, user.id=32d928ad18fd42edbd4e2e88be910eaf, scope.system=all) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:52.218346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83e6a1-13ca-45b9-be05-2ac401f22b56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:52.220904 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83e6a1-13ca-45b9-be05-2ac401f22b56 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:52.230306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 150/312] 10.222.0.79 () {56 vars in 1308 bytes} [Tue Aug 9 16:31:52 2022] GET /identity/v3/role_assignments?user.id=32d928ad18fd42edbd4e2e88be910eaf&scope.system=all&role.id=d7c03cc554a74d0aad88976354649d2d => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:52.290265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71ba6-a185-4cf8-8219-86e859ce0e6c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:52.329486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:52.337983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183547d-9208-4622-8e7c-a04afd33ffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:52.338886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183547d-9208-4622-8e7c-a04afd33ffeb None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:52.339160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183547d-9208-4622-8e7c-a04afd33ffeb None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:52.339423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183547d-9208-4622-8e7c-a04afd33ffeb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:52.353130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183547d-9208-4622-8e7c-a04afd33ffeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-b183547d-9208-4622-8e7c-a04afd33ffeb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:52.358613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:52.361652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:52.363150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 163/313] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:52.371099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83e6a1-13ca-45b9-be05-2ac401f22b56 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:52.478995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83e6a1-13ca-45b9-be05-2ac401f22b56 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:52.505603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:52.520047 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca63b0-c8f8-455e-bd2f-00264f3d9569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:52.520716 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca63b0-c8f8-455e-bd2f-00264f3d9569 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:52.521278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca63b0-c8f8-455e-bd2f-00264f3d9569 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:52.521278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca63b0-c8f8-455e-bd2f-00264f3d9569 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:52.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca63b0-c8f8-455e-bd2f-00264f3d9569 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:52.522322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca63b0-c8f8-455e-bd2f-00264f3d9569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:52.523736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca63b0-c8f8-455e-bd2f-00264f3d9569 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:52.533309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 151/314] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:52.541164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183547d-9208-4622-8e7c-a04afd33ffeb None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:52.662403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183547d-9208-4622-8e7c-a04afd33ffeb None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:52.701184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fb717-10f5-4d68-8aa5-afd1f258b3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:52.701815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fb717-10f5-4d68-8aa5-afd1f258b3c4 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:52.702123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fb717-10f5-4d68-8aa5-afd1f258b3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:52.702394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274fb717-10f5-4d68-8aa5-afd1f258b3c4 None admin] PATH_INFO: `/v3/users/e2790d438a3a49629476be590cd5a46b` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:52.703534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fb717-10f5-4d68-8aa5-afd1f258b3c4 None admin] RBAC: Authorizing `identity:get_user(user_id=e2790d438a3a49629476be590cd5a46b)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:52.704985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274fb717-10f5-4d68-8aa5-afd1f258b3c4 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:52.706949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 164/315] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:52 2022] GET /identity/v3/users/e2790d438a3a49629476be590cd5a46b => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:52.716098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca63b0-c8f8-455e-bd2f-00264f3d9569 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:52.834801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca63b0-c8f8-455e-bd2f-00264f3d9569 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:52.873051 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65861206-f082-40d2-9878-7462f9f2cbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:52.873798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65861206-f082-40d2-9878-7462f9f2cbea None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:52.874279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65861206-f082-40d2-9878-7462f9f2cbea None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:52.874734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65861206-f082-40d2-9878-7462f9f2cbea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:52.943117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:52.985902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65861206-f082-40d2-9878-7462f9f2cbea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:52.987376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65861206-f082-40d2-9878-7462f9f2cbea None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:53.103261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 152/316] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:53.109566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fb717-10f5-4d68-8aa5-afd1f258b3c4 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:53.188116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:53.233698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274fb717-10f5-4d68-8aa5-afd1f258b3c4 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:53.275654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fef09e-bc09-4b94-80cb-18137821d1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:53.276480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fef09e-bc09-4b94-80cb-18137821d1c0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:53.276754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fef09e-bc09-4b94-80cb-18137821d1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:53.277117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fef09e-bc09-4b94-80cb-18137821d1c0 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:53.278332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fef09e-bc09-4b94-80cb-18137821d1c0 None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:53.279756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fef09e-bc09-4b94-80cb-18137821d1c0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:53.281758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 165/317] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:53 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:53.287880 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65861206-f082-40d2-9878-7462f9f2cbea None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:53.361910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:53.401638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65861206-f082-40d2-9878-7462f9f2cbea None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:53.465184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c63ee0-2f14-4ae7-ac9b-adf004311714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:53.466738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c63ee0-2f14-4ae7-ac9b-adf004311714 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:53.467529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c63ee0-2f14-4ae7-ac9b-adf004311714 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:53.468284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c63ee0-2f14-4ae7-ac9b-adf004311714 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:53.469839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c63ee0-2f14-4ae7-ac9b-adf004311714 None admin] RBAC: Adding query filter params (role.id=d7c03cc554a74d0aad88976354649d2d, scope.project.id=5796807ec0ad4cab845934a9df6b9689, user.id=e2790d438a3a49629476be590cd5a46b) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:53.470759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c63ee0-2f14-4ae7-ac9b-adf004311714 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:53.473495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c63ee0-2f14-4ae7-ac9b-adf004311714 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:53.484891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 153/318] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:31:53 2022] GET /identity/v3/role_assignments?user.id=e2790d438a3a49629476be590cd5a46b&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=d7c03cc554a74d0aad88976354649d2d => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:53.516878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e5eed-5cdd-421c-a00b-069d16e5df26 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:53.517299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e5eed-5cdd-421c-a00b-069d16e5df26 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:53.517741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8e5eed-5cdd-421c-a00b-069d16e5df26 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:53.522081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 166/319] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:53.528103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e75dc0-8090-45b1-88de-bbe361bf1575 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:53.528375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e75dc0-8090-45b1-88de-bbe361bf1575 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:53.528637 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e75dc0-8090-45b1-88de-bbe361bf1575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:53.549312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-05e75dc0-8090-45b1-88de-bbe361bf1575 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:53.550567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-05e75dc0-8090-45b1-88de-bbe361bf1575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:53.633238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 154/320] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:53.656923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e5eed-5cdd-421c-a00b-069d16e5df26 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:53.769487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8e5eed-5cdd-421c-a00b-069d16e5df26 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:53.796159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:53.807407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e1ed0-176b-46fc-b013-280ddb59a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:53.808012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e1ed0-176b-46fc-b013-280ddb59a2ce None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:53.808286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e1ed0-176b-46fc-b013-280ddb59a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:53.808589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e1ed0-176b-46fc-b013-280ddb59a2ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:53.809809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e1ed0-176b-46fc-b013-280ddb59a2ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:53.811249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e1ed0-176b-46fc-b013-280ddb59a2ce None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:53.815111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 167/321] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:31:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:53.818375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e75dc0-8090-45b1-88de-bbe361bf1575 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:53.886870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:53.920680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e75dc0-8090-45b1-88de-bbe361bf1575 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:53.955264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2649f-9ac2-4a8e-9670-ded8ac132352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:53.955864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2649f-9ac2-4a8e-9670-ded8ac132352 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:53.956133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2649f-9ac2-4a8e-9670-ded8ac132352 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:53.956397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2649f-9ac2-4a8e-9670-ded8ac132352 None admin] PATH_INFO: `/v3/groups` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:53.957018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2649f-9ac2-4a8e-9670-ded8ac132352 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:53.959003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2649f-9ac2-4a8e-9670-ded8ac132352 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:54.036998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 155/322] 10.222.0.79 () {60 vars in 1134 bytes} [Tue Aug 9 16:31:53 2022] POST /identity/v3/groups => generated 226 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:31:54.437726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795dfb-9cd8-4be2-abbc-261bd0fcc1ac None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:54.438421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795dfb-9cd8-4be2-abbc-261bd0fcc1ac None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:54.438889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16795dfb-9cd8-4be2-abbc-261bd0fcc1ac None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:54.440421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 168/323] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:54.448384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa19ef2-b5ba-4bc2-a12e-3964dccc799f None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:54.448688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa19ef2-b5ba-4bc2-a12e-3964dccc799f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:54.448925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa19ef2-b5ba-4bc2-a12e-3964dccc799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:54.472423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f592eb-c7cf-4536-b496-683207a3f730 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:54.472423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f592eb-c7cf-4536-b496-683207a3f730 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:54.472423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f592eb-c7cf-4536-b496-683207a3f730 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:54.473218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 169/324] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:54.474266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-1fa19ef2-b5ba-4bc2-a12e-3964dccc799f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:54.475670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-1fa19ef2-b5ba-4bc2-a12e-3964dccc799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:54.478281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab71829-6bee-48e9-bfc7-1fc0aaf0be5c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:54.478558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab71829-6bee-48e9-bfc7-1fc0aaf0be5c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:54.478830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab71829-6bee-48e9-bfc7-1fc0aaf0be5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:54.501467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-dab71829-6bee-48e9-bfc7-1fc0aaf0be5c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:54.503872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-dab71829-6bee-48e9-bfc7-1fc0aaf0be5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:54.588141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 156/325] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:54.598311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 170/326] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:54.612146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa19ef2-b5ba-4bc2-a12e-3964dccc799f None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:54.623362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab71829-6bee-48e9-bfc7-1fc0aaf0be5c None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:54.673330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:54.723916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa19ef2-b5ba-4bc2-a12e-3964dccc799f None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:54.735415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab71829-6bee-48e9-bfc7-1fc0aaf0be5c None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:54.761747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d649d-06cb-47fd-b173-a6bf13c6cdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:54.762402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d649d-06cb-47fd-b173-a6bf13c6cdf7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:54.762674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d649d-06cb-47fd-b173-a6bf13c6cdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:54.762935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d649d-06cb-47fd-b173-a6bf13c6cdf7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:54.767428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:54.774417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2087191-c605-4c0c-b199-12c28b05c185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:54.775025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d649d-06cb-47fd-b173-a6bf13c6cdf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:54.775119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2087191-c605-4c0c-b199-12c28b05c185 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:54.775312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2087191-c605-4c0c-b199-12c28b05c185 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:54.775576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2087191-c605-4c0c-b199-12c28b05c185 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-536d649d-06cb-47fd-b173-a6bf13c6cdf7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:54.781746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:54.782986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:54.783814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 157/327] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:54.788145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d649d-06cb-47fd-b173-a6bf13c6cdf7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:54.795741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2087191-c605-4c0c-b199-12c28b05c185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-a2087191-c605-4c0c-b199-12c28b05c185 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:54.802377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:54.804172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:54.806083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:54.811700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 171/328] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:31:54.817658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2087191-c605-4c0c-b199-12c28b05c185 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:54.900373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d649d-06cb-47fd-b173-a6bf13c6cdf7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:54.918093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:54.937775 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bcada-5995-4d39-be60-a7fb5b9a76a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:54.938504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bcada-5995-4d39-be60-a7fb5b9a76a0 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:54.938839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bcada-5995-4d39-be60-a7fb5b9a76a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:54.939169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bcada-5995-4d39-be60-a7fb5b9a76a0 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:54.939909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bcada-5995-4d39-be60-a7fb5b9a76a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:54.940271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bcada-5995-4d39-be60-a7fb5b9a76a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:54.941709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bcada-5995-4d39-be60-a7fb5b9a76a0 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:54.952642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 158/329] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:54.956232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2087191-c605-4c0c-b199-12c28b05c185 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:54.959003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bcada-5995-4d39-be60-a7fb5b9a76a0 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:54.995091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283732e3-3a5f-4f72-ba86-53999ac23676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:54.995616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283732e3-3a5f-4f72-ba86-53999ac23676 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:54.995890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283732e3-3a5f-4f72-ba86-53999ac23676 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:54.996219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283732e3-3a5f-4f72-ba86-53999ac23676 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:54.996889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283732e3-3a5f-4f72-ba86-53999ac23676 None admin] RBAC: Adding query filter params (name=member) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:54.997231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283732e3-3a5f-4f72-ba86-53999ac23676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:54.998657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283732e3-3a5f-4f72-ba86-53999ac23676 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:55.009020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 172/330] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:55.014787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283732e3-3a5f-4f72-ba86-53999ac23676 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:55.065909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bcada-5995-4d39-be60-a7fb5b9a76a0 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:55.074332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:55.102870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb980231-e005-4249-b064-6f3e5674b463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:55.103503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb980231-e005-4249-b064-6f3e5674b463 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:55.103776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb980231-e005-4249-b064-6f3e5674b463 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:55.104040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb980231-e005-4249-b064-6f3e5674b463 None admin] PATH_INFO: `/v3/users/32d928ad18fd42edbd4e2e88be910eaf` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:55.105173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb980231-e005-4249-b064-6f3e5674b463 None admin] RBAC: Authorizing `identity:get_user(user_id=32d928ad18fd42edbd4e2e88be910eaf)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:55.106624 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb980231-e005-4249-b064-6f3e5674b463 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:55.108532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 159/331] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:54 2022] GET /identity/v3/users/32d928ad18fd42edbd4e2e88be910eaf => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:55.113994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb980231-e005-4249-b064-6f3e5674b463 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:55.127538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283732e3-3a5f-4f72-ba86-53999ac23676 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:55.166987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11008d88-3183-4c89-86d1-7406e3fc4c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:55.167618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11008d88-3183-4c89-86d1-7406e3fc4c2e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:55.167895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11008d88-3183-4c89-86d1-7406e3fc4c2e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:55.168159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11008d88-3183-4c89-86d1-7406e3fc4c2e None admin] PATH_INFO: `/v3/users/add7a433ab424e4b9152da210a551eed` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:55.169314 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11008d88-3183-4c89-86d1-7406e3fc4c2e None admin] RBAC: Authorizing `identity:get_user(user_id=add7a433ab424e4b9152da210a551eed)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:55.170779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11008d88-3183-4c89-86d1-7406e3fc4c2e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:55.171212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:55.173438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 173/332] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:31:55 2022] GET /identity/v3/users/add7a433ab424e4b9152da210a551eed => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:55.179914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11008d88-3183-4c89-86d1-7406e3fc4c2e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:55.232294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb980231-e005-4249-b064-6f3e5674b463 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:55.272344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec028249-2b3f-4697-b1aa-8287da680846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:55.272941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec028249-2b3f-4697-b1aa-8287da680846 None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:55.273330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec028249-2b3f-4697-b1aa-8287da680846 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:55.273603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec028249-2b3f-4697-b1aa-8287da680846 None admin] PATH_INFO: `/v3/system/users/32d928ad18fd42edbd4e2e88be910eaf/roles/d7c03cc554a74d0aad88976354649d2d` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:55.275074 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec028249-2b3f-4697-b1aa-8287da680846 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=32d928ad18fd42edbd4e2e88be910eaf, role_id=d7c03cc554a74d0aad88976354649d2d)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:55.276554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec028249-2b3f-4697-b1aa-8287da680846 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:55.289634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11008d88-3183-4c89-86d1-7406e3fc4c2e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:55.303775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:55.325187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96e213-2559-4398-8936-c735278cb2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:55.325832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96e213-2559-4398-8936-c735278cb2bd None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:55.326222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96e213-2559-4398-8936-c735278cb2bd None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:55.326497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96e213-2559-4398-8936-c735278cb2bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:55.327289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96e213-2559-4398-8936-c735278cb2bd None admin] RBAC: Adding query filter params (role.id=2490390ea61f4cf594df3587e7674650, user.id=add7a433ab424e4b9152da210a551eed, scope.system=all) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:55.327648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96e213-2559-4398-8936-c735278cb2bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:55.329073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96e213-2559-4398-8936-c735278cb2bd None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:55.342422 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 174/333] 10.222.0.79 () {56 vars in 1308 bytes} [Tue Aug 9 16:31:55 2022] GET /identity/v3/role_assignments?user.id=add7a433ab424e4b9152da210a551eed&scope.system=all&role.id=2490390ea61f4cf594df3587e7674650 => generated 510 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:55.368020 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 160/334] 10.222.0.79 () {58 vars in 1334 bytes} [Tue Aug 9 16:31:55 2022] PUT /identity/v3/system/users/32d928ad18fd42edbd4e2e88be910eaf/roles/d7c03cc554a74d0aad88976354649d2d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:31:56.972943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c27dfa-5aa1-4076-b409-c87079df77da None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:56.974006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c27dfa-5aa1-4076-b409-c87079df77da None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:56.975019 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c27dfa-5aa1-4076-b409-c87079df77da None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:56.976862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 175/335] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:56.983991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe4a21-3fbd-4a70-9243-9e629e31cc89 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:56.984905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe4a21-3fbd-4a70-9243-9e629e31cc89 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:56.985379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe4a21-3fbd-4a70-9243-9e629e31cc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:57.007574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-41fe4a21-3fbd-4a70-9243-9e629e31cc89 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:57.009087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-41fe4a21-3fbd-4a70-9243-9e629e31cc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:57.052434 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ba611-c527-451a-bef9-7d9bba229e20 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:57.053069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ba611-c527-451a-bef9-7d9bba229e20 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:57.053716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ba611-c527-451a-bef9-7d9bba229e20 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:57.056632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 176/336] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:57.062338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c50af-d374-4fc3-94f2-f65a33cefc35 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:57.062920 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c50af-d374-4fc3-94f2-f65a33cefc35 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:57.063388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c50af-d374-4fc3-94f2-f65a33cefc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:57.096078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ab0c50af-d374-4fc3-94f2-f65a33cefc35 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:57.097639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ab0c50af-d374-4fc3-94f2-f65a33cefc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:57.101965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 161/337] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:57.139340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe4a21-3fbd-4a70-9243-9e629e31cc89 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:57.195593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 177/338] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:57.219094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c50af-d374-4fc3-94f2-f65a33cefc35 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:57.222393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:57.289177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe4a21-3fbd-4a70-9243-9e629e31cc89 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:57.344561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df8acb-a8ba-4058-9878-e319b7ab2bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:57.345193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8acb-a8ba-4058-9878-e319b7ab2bf7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:57.345466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8acb-a8ba-4058-9878-e319b7ab2bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:57.345732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8acb-a8ba-4058-9878-e319b7ab2bf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:57.363632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df8acb-a8ba-4058-9878-e319b7ab2bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-b8df8acb-a8ba-4058-9878-e319b7ab2bf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:57.368605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:57.372064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:57.372870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:57.374282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 162/339] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:31:57.375591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:57.375955 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df8acb-a8ba-4058-9878-e319b7ab2bf7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:57.385119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c50af-d374-4fc3-94f2-f65a33cefc35 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:57.470204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34788e-7bb2-43fe-8ce0-43918666934e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:57.471190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34788e-7bb2-43fe-8ce0-43918666934e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:57.471668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34788e-7bb2-43fe-8ce0-43918666934e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:57.472161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34788e-7bb2-43fe-8ce0-43918666934e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:57.498990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b34788e-7bb2-43fe-8ce0-43918666934e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-1b34788e-7bb2-43fe-8ce0-43918666934e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:57.504586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:57.506096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:57.506935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:57.509644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 178/340] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:31:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:31:57.515095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34788e-7bb2-43fe-8ce0-43918666934e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:57.527218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df8acb-a8ba-4058-9878-e319b7ab2bf7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:57.539847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:57.580121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e22d4-7f44-407e-83f4-8d8a1c874e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:57.580732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e22d4-7f44-407e-83f4-8d8a1c874e02 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:57.581003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e22d4-7f44-407e-83f4-8d8a1c874e02 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:57.581276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e22d4-7f44-407e-83f4-8d8a1c874e02 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:57.581932 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e22d4-7f44-407e-83f4-8d8a1c874e02 None admin] RBAC: Adding query filter params (name=member) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:57.582246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e22d4-7f44-407e-83f4-8d8a1c874e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:57.584167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6e22d4-7f44-407e-83f4-8d8a1c874e02 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:57.598441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 163/341] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:57.603853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e22d4-7f44-407e-83f4-8d8a1c874e02 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:57.713127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34788e-7bb2-43fe-8ce0-43918666934e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:57.757757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9854b97-a27c-4149-b65d-89e945e7d03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:57.758403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9854b97-a27c-4149-b65d-89e945e7d03e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:57.758679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9854b97-a27c-4149-b65d-89e945e7d03e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:57.758948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9854b97-a27c-4149-b65d-89e945e7d03e None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:57.759622 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9854b97-a27c-4149-b65d-89e945e7d03e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:57.759921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9854b97-a27c-4149-b65d-89e945e7d03e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:57.761406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9854b97-a27c-4149-b65d-89e945e7d03e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:57.770364 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 179/342] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:31:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:31:57.771947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d9c43-d01e-42f4-a72c-13c8ce461887 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:57.772259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d9c43-d01e-42f4-a72c-13c8ce461887 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:57.772534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d9c43-d01e-42f4-a72c-13c8ce461887 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:57.778413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 180/343] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:57.779907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d9c43-d01e-42f4-a72c-13c8ce461887 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:57.792621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e22d4-7f44-407e-83f4-8d8a1c874e02 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:57.807333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:57.877193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db06e2-7f45-4f43-a2e3-a73671b17603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:57.877819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db06e2-7f45-4f43-a2e3-a73671b17603 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:57.878126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db06e2-7f45-4f43-a2e3-a73671b17603 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:57.878399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db06e2-7f45-4f43-a2e3-a73671b17603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:57.896018 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d9c43-d01e-42f4-a72c-13c8ce461887 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:57.938417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:57.940776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676e302-7602-4dc9-8f0d-1ddbf5fb4a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:57.942147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676e302-7602-4dc9-8f0d-1ddbf5fb4a53 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:57.942811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676e302-7602-4dc9-8f0d-1ddbf5fb4a53 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:57.944706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676e302-7602-4dc9-8f0d-1ddbf5fb4a53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.024045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db06e2-7f45-4f43-a2e3-a73671b17603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:58.026375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db06e2-7f45-4f43-a2e3-a73671b17603 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:58.081463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676e302-7602-4dc9-8f0d-1ddbf5fb4a53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:58.083724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676e302-7602-4dc9-8f0d-1ddbf5fb4a53 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:58.133412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:58.187443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 164/344] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:58.188943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448f8a-da60-4f69-8476-41b70b653533 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:58.189371 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448f8a-da60-4f69-8476-41b70b653533 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:58.189774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5448f8a-da60-4f69-8476-41b70b653533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.210729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d5448f8a-da60-4f69-8476-41b70b653533 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:58.212153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d5448f8a-da60-4f69-8476-41b70b653533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:58.293849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 165/345] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:58.294300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:58.296181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 181/346] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 16:31:58.301046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448f8a-da60-4f69-8476-41b70b653533 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:58.301919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676e302-7602-4dc9-8f0d-1ddbf5fb4a53 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:58.413460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5448f8a-da60-4f69-8476-41b70b653533 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:58.454471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1a699-1665-4ecd-bc49-6362bfd18f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:58.455354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1a699-1665-4ecd-bc49-6362bfd18f12 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:58.455897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1a699-1665-4ecd-bc49-6362bfd18f12 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:58.456303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1a699-1665-4ecd-bc49-6362bfd18f12 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.457726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1a699-1665-4ecd-bc49-6362bfd18f12 None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:58.459953 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f1a699-1665-4ecd-bc49-6362bfd18f12 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:58.461982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 166/347] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:58 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:58.463030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1a699-1665-4ecd-bc49-6362bfd18f12 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:58.464442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676e302-7602-4dc9-8f0d-1ddbf5fb4a53 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:58.508715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:58.526256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69750ee-cde8-45df-a46c-3b4fc8988df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:58.527248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69750ee-cde8-45df-a46c-3b4fc8988df0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:58.527758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69750ee-cde8-45df-a46c-3b4fc8988df0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:58.528361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69750ee-cde8-45df-a46c-3b4fc8988df0 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.530596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69750ee-cde8-45df-a46c-3b4fc8988df0 None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:58.534850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69750ee-cde8-45df-a46c-3b4fc8988df0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:58.535258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 182/348] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:58 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:58.536190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69750ee-cde8-45df-a46c-3b4fc8988df0 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:58.554292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:58.603016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f1a699-1665-4ecd-bc49-6362bfd18f12 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:58.651504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7d21e-9422-4ca1-8eb5-9d1866dae6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:58.652739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7d21e-9422-4ca1-8eb5-9d1866dae6ff None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:58.653195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7d21e-9422-4ca1-8eb5-9d1866dae6ff None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:58.653685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7d21e-9422-4ca1-8eb5-9d1866dae6ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.665717 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69750ee-cde8-45df-a46c-3b4fc8988df0 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:58.669910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7d21e-9422-4ca1-8eb5-9d1866dae6ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-97c7d21e-9422-4ca1-8eb5-9d1866dae6ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:58.675345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:58.676511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:58.677521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 167/349] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:58.678214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e167c2-a99c-48a2-ba4e-0df754063563 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:58.678214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e167c2-a99c-48a2-ba4e-0df754063563 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:58.678344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e167c2-a99c-48a2-ba4e-0df754063563 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.679663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 168/350] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:58.682754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e167c2-a99c-48a2-ba4e-0df754063563 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:58.720266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0bbfd-7b8e-4131-9739-11404ee7f2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:58.721177 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0bbfd-7b8e-4131-9739-11404ee7f2e5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:58.721650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0bbfd-7b8e-4131-9739-11404ee7f2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:58.722103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0bbfd-7b8e-4131-9739-11404ee7f2e5 None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.729631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0bbfd-7b8e-4131-9739-11404ee7f2e5 None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:58.731393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0bbfd-7b8e-4131-9739-11404ee7f2e5 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:58.733419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 183/351] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:31:58 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:58.736724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0bbfd-7b8e-4131-9739-11404ee7f2e5 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:58.811536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:58.823025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e167c2-a99c-48a2-ba4e-0df754063563 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:58.850270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:58.873657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ffaf2-5ade-405a-8ae2-015e2ada6baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:58.874327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ffaf2-5ade-405a-8ae2-015e2ada6baf None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:58.874605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ffaf2-5ade-405a-8ae2-015e2ada6baf None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:58.874876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ffaf2-5ade-405a-8ae2-015e2ada6baf None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.876725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ffaf2-5ade-405a-8ae2-015e2ada6baf None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:58.878251 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ffaf2-5ade-405a-8ae2-015e2ada6baf None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:58.880124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 169/352] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:31:58 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:58.881799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9d042-fc03-47d1-b04e-e71ecd486225 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:58.884959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9d042-fc03-47d1-b04e-e71ecd486225 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:58.884959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9d042-fc03-47d1-b04e-e71ecd486225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.891266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0bbfd-7b8e-4131-9739-11404ee7f2e5 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:58.906832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-65e9d042-fc03-47d1-b04e-e71ecd486225 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:58.908075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-65e9d042-fc03-47d1-b04e-e71ecd486225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:58.932916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc45732-ae81-4781-a3d8-12606cc0a164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:58.933436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc45732-ae81-4781-a3d8-12606cc0a164 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:58.933712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc45732-ae81-4781-a3d8-12606cc0a164 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:58.934055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc45732-ae81-4781-a3d8-12606cc0a164 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.934683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc45732-ae81-4781-a3d8-12606cc0a164 None admin] RBAC: Adding query filter params (name=member) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:58.934981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc45732-ae81-4781-a3d8-12606cc0a164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:58.936403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc45732-ae81-4781-a3d8-12606cc0a164 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:58.946181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 184/353] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:31:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:58.946841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc45732-ae81-4781-a3d8-12606cc0a164 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:58.995120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 170/354] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:58.996656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d187f90-559c-4899-a199-7a8e8cf6685e None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:58.996942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d187f90-559c-4899-a199-7a8e8cf6685e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:58.997210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d187f90-559c-4899-a199-7a8e8cf6685e None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:58.999668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 171/355] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:59.000439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d187f90-559c-4899-a199-7a8e8cf6685e None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:59.075998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc45732-ae81-4781-a3d8-12606cc0a164 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:59.096957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:59.119554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f7c01-3a32-4a76-a65f-06ed11b3a323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:59.120103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f7c01-3a32-4a76-a65f-06ed11b3a323 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:59.120806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f7c01-3a32-4a76-a65f-06ed11b3a323 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:59.120806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f7c01-3a32-4a76-a65f-06ed11b3a323 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:59.122176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f7c01-3a32-4a76-a65f-06ed11b3a323 None admin] RBAC: Adding query filter params (group.id=9908b486ccf84e6ea374363f9ae8c5e5, role.id=2490390ea61f4cf594df3587e7674650, scope.project.id=28843c17667d4709ab63a1378bc693a1) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:59.122176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f7c01-3a32-4a76-a65f-06ed11b3a323 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:59.123322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387f7c01-3a32-4a76-a65f-06ed11b3a323 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:59.131357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 185/356] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Aug 9 16:31:58 2022] GET /identity/v3/role_assignments?group.id=9908b486ccf84e6ea374363f9ae8c5e5&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=2490390ea61f4cf594df3587e7674650 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:59.132741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e95981-87a6-4565-a543-ba6ffc3b4463 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:59.133026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e95981-87a6-4565-a543-ba6ffc3b4463 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:59.133299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e95981-87a6-4565-a543-ba6ffc3b4463 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:59.134734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 186/357] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:31:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:31:59.135544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e95981-87a6-4565-a543-ba6ffc3b4463 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:59.150781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d187f90-559c-4899-a199-7a8e8cf6685e None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:59.176732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:59.214670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66366659-df7e-4572-b925-f20e004b1b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:59.215204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66366659-df7e-4572-b925-f20e004b1b61 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:59.215480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66366659-df7e-4572-b925-f20e004b1b61 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:59.215743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66366659-df7e-4572-b925-f20e004b1b61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:59.216480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66366659-df7e-4572-b925-f20e004b1b61 None admin] RBAC: Adding query filter params (group.id=9908b486ccf84e6ea374363f9ae8c5e5, role.id=20d35b10934447938237c0db5202bbfb, scope.project.id=28843c17667d4709ab63a1378bc693a1) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:59.216786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66366659-df7e-4572-b925-f20e004b1b61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:59.218330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66366659-df7e-4572-b925-f20e004b1b61 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:59.227320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 172/358] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Aug 9 16:31:58 2022] GET /identity/v3/role_assignments?group.id=9908b486ccf84e6ea374363f9ae8c5e5&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=20d35b10934447938237c0db5202bbfb => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:59.229272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206151c-c267-4b47-935f-9dfcf53d4759 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:59.229272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206151c-c267-4b47-935f-9dfcf53d4759 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:59.229272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206151c-c267-4b47-935f-9dfcf53d4759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:59.254317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9206151c-c267-4b47-935f-9dfcf53d4759 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:59.256170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9206151c-c267-4b47-935f-9dfcf53d4759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:59.296841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e95981-87a6-4565-a543-ba6ffc3b4463 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:59.349447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c7161-263d-49b9-9130-549cfab65b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:59.350203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c7161-263d-49b9-9130-549cfab65b89 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:59.350637 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 173/359] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:59.350753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c7161-263d-49b9-9130-549cfab65b89 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:59.350938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c7161-263d-49b9-9130-549cfab65b89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:59.351789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206151c-c267-4b47-935f-9dfcf53d4759 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:59.424077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:59.457819 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c7161-263d-49b9-9130-549cfab65b89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:59.460166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c7161-263d-49b9-9130-549cfab65b89 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:59.483414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206151c-c267-4b47-935f-9dfcf53d4759 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:59.500653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:59.538646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd0d52-4c86-406c-aa78-519c5c4818b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:59.539563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd0d52-4c86-406c-aa78-519c5c4818b2 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:59.539852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd0d52-4c86-406c-aa78-519c5c4818b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:59.540124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd0d52-4c86-406c-aa78-519c5c4818b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:59.559399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd0d52-4c86-406c-aa78-519c5c4818b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-23cd0d52-4c86-406c-aa78-519c5c4818b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:59.564673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:59.565687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:31:59.566512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 174/360] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:31:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:59.567831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c1c65-13c0-4810-baff-a2ae3cd77242 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:59.568125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c1c65-13c0-4810-baff-a2ae3cd77242 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:59.568396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c1c65-13c0-4810-baff-a2ae3cd77242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:59.595815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-201c1c65-13c0-4810-baff-a2ae3cd77242 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:31:59.597786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-201c1c65-13c0-4810-baff-a2ae3cd77242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:31:59.611496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 187/361] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:31:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:31:59.612269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c7161-263d-49b9-9130-549cfab65b89 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:59.696917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 175/362] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:31:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:31:59.697785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c1c65-13c0-4810-baff-a2ae3cd77242 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:59.716448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:59.745094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c7161-263d-49b9-9130-549cfab65b89 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:59.787937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437b763-c265-42c4-8be0-e8b8fc7c8259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:59.788508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437b763-c265-42c4-8be0-e8b8fc7c8259 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:59.788784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437b763-c265-42c4-8be0-e8b8fc7c8259 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:59.789107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437b763-c265-42c4-8be0-e8b8fc7c8259 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:59.800263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437b763-c265-42c4-8be0-e8b8fc7c8259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-c437b763-c265-42c4-8be0-e8b8fc7c8259 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:31:59.806887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:31:59.807807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 188/363] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:31:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:31:59.809002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437b763-c265-42c4-8be0-e8b8fc7c8259 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:59.836942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c1c65-13c0-4810-baff-a2ae3cd77242 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:59.838850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:31:59.876207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb607f47-5c38-44b8-8e39-3dac004fd77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:59.877286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb607f47-5c38-44b8-8e39-3dac004fd77c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:59.877757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb607f47-5c38-44b8-8e39-3dac004fd77c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:59.878237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb607f47-5c38-44b8-8e39-3dac004fd77c None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:59.879193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb607f47-5c38-44b8-8e39-3dac004fd77c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:31:59.879751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb607f47-5c38-44b8-8e39-3dac004fd77c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:59.881359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb607f47-5c38-44b8-8e39-3dac004fd77c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:59.890725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 176/364] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:31:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:59.891985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb607f47-5c38-44b8-8e39-3dac004fd77c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:59.919481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437b763-c265-42c4-8be0-e8b8fc7c8259 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:31:59.959665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df68409-73d9-43de-a1c9-3267d174756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:31:59.960304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df68409-73d9-43de-a1c9-3267d174756a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:31:59.960585 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df68409-73d9-43de-a1c9-3267d174756a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:31:59.962574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df68409-73d9-43de-a1c9-3267d174756a None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:31:59.963834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df68409-73d9-43de-a1c9-3267d174756a None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:31:59.968094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df68409-73d9-43de-a1c9-3267d174756a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:31:59.969401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 189/365] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:31:59 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:31:59.970059 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df68409-73d9-43de-a1c9-3267d174756a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:31:59.986033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:00.012048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb607f47-5c38-44b8-8e39-3dac004fd77c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:00.051536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c0b5a-6d41-46ee-a1ee-ef40676e24c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:00.052565 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c0b5a-6d41-46ee-a1ee-ef40676e24c9 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:00.053035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c0b5a-6d41-46ee-a1ee-ef40676e24c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:00.053515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c0b5a-6d41-46ee-a1ee-ef40676e24c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:00.065715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c0b5a-6d41-46ee-a1ee-ef40676e24c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-280c0b5a-6d41-46ee-a1ee-ef40676e24c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:00.072031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:00.072931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 09 16:32:00.073702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:00.076147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 177/366] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:00.077194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c0b5a-6d41-46ee-a1ee-ef40676e24c9 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:00.086591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df68409-73d9-43de-a1c9-3267d174756a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:00.106068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:00.124318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3fda8-5ab8-4cc7-a944-61263243fe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:00.124941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3fda8-5ab8-4cc7-a944-61263243fe0b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:00.125194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3fda8-5ab8-4cc7-a944-61263243fe0b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:00.125526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3fda8-5ab8-4cc7-a944-61263243fe0b None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:00.126228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3fda8-5ab8-4cc7-a944-61263243fe0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:00.126548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3fda8-5ab8-4cc7-a944-61263243fe0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:00.127970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3fda8-5ab8-4cc7-a944-61263243fe0b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:00.138691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 190/367] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:31:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:00.139635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3fda8-5ab8-4cc7-a944-61263243fe0b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:00.194679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c0b5a-6d41-46ee-a1ee-ef40676e24c9 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:00.226336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:00.232585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb996d-fcba-4df3-8983-a29a30dbdf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:00.233502 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb996d-fcba-4df3-8983-a29a30dbdf99 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:00.234016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb996d-fcba-4df3-8983-a29a30dbdf99 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:00.234480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb996d-fcba-4df3-8983-a29a30dbdf99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:00.249834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3fda8-5ab8-4cc7-a944-61263243fe0b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:00.287760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab9d3e-8a34-4996-b13b-bcf334951196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:00.288440 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab9d3e-8a34-4996-b13b-bcf334951196 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:00.288619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab9d3e-8a34-4996-b13b-bcf334951196 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:00.288899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab9d3e-8a34-4996-b13b-bcf334951196 None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:00.290154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab9d3e-8a34-4996-b13b-bcf334951196 None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:00.291592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab9d3e-8a34-4996-b13b-bcf334951196 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:00.293542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 191/368] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:00 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:00.294412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab9d3e-8a34-4996-b13b-bcf334951196 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:00.309614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb996d-fcba-4df3-8983-a29a30dbdf99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:00.311479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb996d-fcba-4df3-8983-a29a30dbdf99 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:00.339212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:00.409148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab9d3e-8a34-4996-b13b-bcf334951196 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:00.428290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 178/369] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:00.429475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb996d-fcba-4df3-8983-a29a30dbdf99 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:00.457050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951f286-9c5a-4329-940f-bd8abe41219b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:00.457656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951f286-9c5a-4329-940f-bd8abe41219b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:00.458022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951f286-9c5a-4329-940f-bd8abe41219b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:00.458277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e951f286-9c5a-4329-940f-bd8abe41219b None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:00.459512 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951f286-9c5a-4329-940f-bd8abe41219b None admin] RBAC: Adding query filter params (name=reader) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:00.459814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951f286-9c5a-4329-940f-bd8abe41219b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:00.461183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e951f286-9c5a-4329-940f-bd8abe41219b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:00.468764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:00.478336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 192/370] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:32:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:00.479361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951f286-9c5a-4329-940f-bd8abe41219b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:00.557873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb996d-fcba-4df3-8983-a29a30dbdf99 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:00.583448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:00.596388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93459b59-756c-4e69-88e0-30d91e276991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:00.597005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93459b59-756c-4e69-88e0-30d91e276991 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:00.597288 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93459b59-756c-4e69-88e0-30d91e276991 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:00.597562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93459b59-756c-4e69-88e0-30d91e276991 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:00.597725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e951f286-9c5a-4329-940f-bd8abe41219b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:00.638175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd04b7-273b-49e2-8d04-eddbc448e414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:00.638824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd04b7-273b-49e2-8d04-eddbc448e414 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:00.639107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd04b7-273b-49e2-8d04-eddbc448e414 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:00.639380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd04b7-273b-49e2-8d04-eddbc448e414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:00.640104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd04b7-273b-49e2-8d04-eddbc448e414 None admin] RBAC: Adding query filter params (group.id=9908b486ccf84e6ea374363f9ae8c5e5, role.id=2490390ea61f4cf594df3587e7674650, scope.project.id=5796807ec0ad4cab845934a9df6b9689) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:00.640404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd04b7-273b-49e2-8d04-eddbc448e414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:00.641840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd04b7-273b-49e2-8d04-eddbc448e414 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:00.649046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 193/371] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Aug 9 16:32:00 2022] GET /identity/v3/role_assignments?group.id=9908b486ccf84e6ea374363f9ae8c5e5&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=2490390ea61f4cf594df3587e7674650 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:00.649916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd04b7-273b-49e2-8d04-eddbc448e414 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:00.682550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93459b59-756c-4e69-88e0-30d91e276991 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:00.684829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93459b59-756c-4e69-88e0-30d91e276991 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:00.815204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 179/372] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:00.816231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93459b59-756c-4e69-88e0-30d91e276991 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:00.855303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:00.866320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:00.932578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd04b7-273b-49e2-8d04-eddbc448e414 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:00.960286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93459b59-756c-4e69-88e0-30d91e276991 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:00.978474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205c4de-cd5d-4b1f-9422-b04e83151d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:00.981917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205c4de-cd5d-4b1f-9422-b04e83151d2d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:00.983214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205c4de-cd5d-4b1f-9422-b04e83151d2d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:00.983688 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205c4de-cd5d-4b1f-9422-b04e83151d2d None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:00.988601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205c4de-cd5d-4b1f-9422-b04e83151d2d None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:00.990505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205c4de-cd5d-4b1f-9422-b04e83151d2d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:00.995594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 194/373] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:00 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:00.997149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205c4de-cd5d-4b1f-9422-b04e83151d2d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:01.051663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262d6d5-5726-45c6-943c-d5ec16a135f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:01.052661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262d6d5-5726-45c6-943c-d5ec16a135f2 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:01.053077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262d6d5-5726-45c6-943c-d5ec16a135f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:01.053484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8262d6d5-5726-45c6-943c-d5ec16a135f2 None admin] PATH_INFO: `/v3/users/32d928ad18fd42edbd4e2e88be910eaf` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:01.054989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262d6d5-5726-45c6-943c-d5ec16a135f2 None admin] RBAC: Authorizing `identity:get_user(user_id=32d928ad18fd42edbd4e2e88be910eaf)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:01.056638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8262d6d5-5726-45c6-943c-d5ec16a135f2 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:01.058718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 180/374] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:32:00 2022] GET /identity/v3/users/32d928ad18fd42edbd4e2e88be910eaf => generated 293 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:01.061968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262d6d5-5726-45c6-943c-d5ec16a135f2 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:01.166889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205c4de-cd5d-4b1f-9422-b04e83151d2d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:01.198877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:01.205121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:01.211705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8262d6d5-5726-45c6-943c-d5ec16a135f2 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:01.220460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a3fdf-1a3c-4f30-b15f-d2e4d8a34263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:01.220834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a3fdf-1a3c-4f30-b15f-d2e4d8a34263 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:01.221108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a3fdf-1a3c-4f30-b15f-d2e4d8a34263 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:01.221375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a3fdf-1a3c-4f30-b15f-d2e4d8a34263 None admin] PATH_INFO: `/v3/projects/e0a53b8ab06c430ea7305247e9df6f92` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:01.226657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a3fdf-1a3c-4f30-b15f-d2e4d8a34263 None admin] RBAC: Authorizing `identity:get_project(project_id=e0a53b8ab06c430ea7305247e9df6f92)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:01.228186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63a3fdf-1a3c-4f30-b15f-d2e4d8a34263 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:01.229907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 195/375] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:00 2022] GET /identity/v3/projects/e0a53b8ab06c430ea7305247e9df6f92 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:01.230527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a3fdf-1a3c-4f30-b15f-d2e4d8a34263 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:01.254558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c939f-4b08-44c7-8440-ae4668286046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:01.256050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c939f-4b08-44c7-8440-ae4668286046 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:01.256604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c939f-4b08-44c7-8440-ae4668286046 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:01.257062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c939f-4b08-44c7-8440-ae4668286046 None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:01.258816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c939f-4b08-44c7-8440-ae4668286046 None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:01.260451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c939f-4b08-44c7-8440-ae4668286046 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:01.262859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 181/376] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:01 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:01.264463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c939f-4b08-44c7-8440-ae4668286046 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:01.405302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c939f-4b08-44c7-8440-ae4668286046 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:01.436807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a3fdf-1a3c-4f30-b15f-d2e4d8a34263 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:01.482282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfdff8-f6e4-4136-a849-3a2e740816ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:01.482910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfdff8-f6e4-4136-a849-3a2e740816ee None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:01.483182 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfdff8-f6e4-4136-a849-3a2e740816ee None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:01.483445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfdff8-f6e4-4136-a849-3a2e740816ee None admin] PATH_INFO: `/v3/groups/ae7fd0887fc946ec8845d5b128694719` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:01.497324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfdff8-f6e4-4136-a849-3a2e740816ee None admin] RBAC: Authorizing `identity:get_group(group_id=ae7fd0887fc946ec8845d5b128694719)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:01.498783 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfdff8-f6e4-4136-a849-3a2e740816ee None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:01.504073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 182/377] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:01 2022] GET /identity/v3/groups/ae7fd0887fc946ec8845d5b128694719 => generated 229 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:01.504883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfdff8-f6e4-4136-a849-3a2e740816ee None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:01.514377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b5685-a67d-495d-860e-b4d871335d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:01.515049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b5685-a67d-495d-860e-b4d871335d42 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:01.515250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b5685-a67d-495d-860e-b4d871335d42 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:01.515753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b5685-a67d-495d-860e-b4d871335d42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:01.516236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b5685-a67d-495d-860e-b4d871335d42 None admin] RBAC: Adding query filter params (role.id=d7c03cc554a74d0aad88976354649d2d, user.id=32d928ad18fd42edbd4e2e88be910eaf, scope.system=all) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:01.516538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b5685-a67d-495d-860e-b4d871335d42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:01.518168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b5685-a67d-495d-860e-b4d871335d42 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:01.546236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 196/378] 10.222.0.79 () {56 vars in 1308 bytes} [Tue Aug 9 16:32:01 2022] GET /identity/v3/role_assignments?user.id=32d928ad18fd42edbd4e2e88be910eaf&scope.system=all&role.id=d7c03cc554a74d0aad88976354649d2d => generated 510 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:01.547135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b5685-a67d-495d-860e-b4d871335d42 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:01.572347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:01.639749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:01.690269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b5685-a67d-495d-860e-b4d871335d42 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:01.740041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b16c0f-b5b4-449e-a984-e0ae76248135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:01.740657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b16c0f-b5b4-449e-a984-e0ae76248135 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:01.740927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b16c0f-b5b4-449e-a984-e0ae76248135 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:01.741189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b16c0f-b5b4-449e-a984-e0ae76248135 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:01.741367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfdff8-f6e4-4136-a849-3a2e740816ee None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:01.741920 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b16c0f-b5b4-449e-a984-e0ae76248135 None admin] RBAC: Adding query filter params (group.id=ae7fd0887fc946ec8845d5b128694719, role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=e0a53b8ab06c430ea7305247e9df6f92) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:01.742274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b16c0f-b5b4-449e-a984-e0ae76248135 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:01.744010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b16c0f-b5b4-449e-a984-e0ae76248135 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:01.758407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 197/379] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Aug 9 16:32:01 2022] GET /identity/v3/role_assignments?group.id=ae7fd0887fc946ec8845d5b128694719&scope.project.id=e0a53b8ab06c430ea7305247e9df6f92&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:01.795878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a411-ce4a-48fc-9224-ecf55d6a467a None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:01.796215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a411-ce4a-48fc-9224-ecf55d6a467a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:01.796420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494a411-ce4a-48fc-9224-ecf55d6a467a None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:01.797801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 198/380] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:01.810233 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9946-a135-42ea-81c5-217d0c6f1a71 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:01.810566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9946-a135-42ea-81c5-217d0c6f1a71 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:01.810770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f9946-a135-42ea-81c5-217d0c6f1a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:01.835294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b132422-3650-48ff-bb92-1c13394a6098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:01.835930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b132422-3650-48ff-bb92-1c13394a6098 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:01.836213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b132422-3650-48ff-bb92-1c13394a6098 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:01.836478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b132422-3650-48ff-bb92-1c13394a6098 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:01.837232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b132422-3650-48ff-bb92-1c13394a6098 None admin] RBAC: Adding query filter params (group.id=9908b486ccf84e6ea374363f9ae8c5e5, role.id=20d35b10934447938237c0db5202bbfb, scope.project.id=5796807ec0ad4cab845934a9df6b9689) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:01.837536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b132422-3650-48ff-bb92-1c13394a6098 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:01.839037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b132422-3650-48ff-bb92-1c13394a6098 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:01.843041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-f93f9946-a135-42ea-81c5-217d0c6f1a71 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:01.844706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-f93f9946-a135-42ea-81c5-217d0c6f1a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:01.845670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 183/381] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Aug 9 16:32:01 2022] GET /identity/v3/role_assignments?group.id=9908b486ccf84e6ea374363f9ae8c5e5&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=20d35b10934447938237c0db5202bbfb => generated 260 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:01.939585 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 199/382] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:32:01.971850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b132422-3650-48ff-bb92-1c13394a6098 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:01.988409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d03142-3afd-4e79-a929-f0a6230dd0e6 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:01.988733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d03142-3afd-4e79-a929-f0a6230dd0e6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:01.988949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d03142-3afd-4e79-a929-f0a6230dd0e6 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:01.990721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 200/383] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:02.028301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad273c3-100f-490b-b54e-daad3385a5dd None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:02.028640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad273c3-100f-490b-b54e-daad3385a5dd None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:02.028952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad273c3-100f-490b-b54e-daad3385a5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:02.058986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-4ad273c3-100f-490b-b54e-daad3385a5dd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:02.060338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4ad273c3-100f-490b-b54e-daad3385a5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:02.088203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:02.106168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b132422-3650-48ff-bb92-1c13394a6098 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:02.143532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:02.149328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25253dbe-3f63-4e09-98bc-2071637ab333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:02.150158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25253dbe-3f63-4e09-98bc-2071637ab333 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:02.150561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25253dbe-3f63-4e09-98bc-2071637ab333 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:02.150833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25253dbe-3f63-4e09-98bc-2071637ab333 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:02.164045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 201/384] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:02.167798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25253dbe-3f63-4e09-98bc-2071637ab333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-25253dbe-3f63-4e09-98bc-2071637ab333 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:02.173263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:02.174225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:02.175093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 184/385] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:02.182499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad273c3-100f-490b-b54e-daad3385a5dd None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:02.189408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25253dbe-3f63-4e09-98bc-2071637ab333 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:02.360970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad273c3-100f-490b-b54e-daad3385a5dd None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:02.402943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2991a67-7e76-451f-91fb-3e1f5d7d11b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:02.403556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2991a67-7e76-451f-91fb-3e1f5d7d11b6 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:02.403832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2991a67-7e76-451f-91fb-3e1f5d7d11b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:02.404095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2991a67-7e76-451f-91fb-3e1f5d7d11b6 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:02.404763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2991a67-7e76-451f-91fb-3e1f5d7d11b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:02.405063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2991a67-7e76-451f-91fb-3e1f5d7d11b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:02.406484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2991a67-7e76-451f-91fb-3e1f5d7d11b6 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:02.420942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 202/386] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:32:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:02.422451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2991a67-7e76-451f-91fb-3e1f5d7d11b6 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:02.479854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25253dbe-3f63-4e09-98bc-2071637ab333 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:02.483650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:02.525978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0898222-d7d8-43aa-b3e4-be2944dce197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:02.525978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0898222-d7d8-43aa-b3e4-be2944dce197 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:02.525978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0898222-d7d8-43aa-b3e4-be2944dce197 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:02.525978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0898222-d7d8-43aa-b3e4-be2944dce197 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:02.538650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0898222-d7d8-43aa-b3e4-be2944dce197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-b0898222-d7d8-43aa-b3e4-be2944dce197 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:02.543971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:02.544876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:02.545668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 185/387] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:02.554947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0898222-d7d8-43aa-b3e4-be2944dce197 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:02.566826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2991a67-7e76-451f-91fb-3e1f5d7d11b6 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:02.568287 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:02.609205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0ecb4-9476-4a61-80b8-0cfaeff05273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:02.609877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0ecb4-9476-4a61-80b8-0cfaeff05273 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:02.610199 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0ecb4-9476-4a61-80b8-0cfaeff05273 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:02.610464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0ecb4-9476-4a61-80b8-0cfaeff05273 None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:02.611993 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0ecb4-9476-4a61-80b8-0cfaeff05273 None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:02.613436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0ecb4-9476-4a61-80b8-0cfaeff05273 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:02.615956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 203/388] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:02 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:02.638963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0ecb4-9476-4a61-80b8-0cfaeff05273 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:02.676288 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0898222-d7d8-43aa-b3e4-be2944dce197 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:02.711089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b16a95-d13c-494d-88f9-b7bab2ffddbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:02.711691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b16a95-d13c-494d-88f9-b7bab2ffddbc None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:02.711966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b16a95-d13c-494d-88f9-b7bab2ffddbc None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:02.712228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b16a95-d13c-494d-88f9-b7bab2ffddbc None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:02.712886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b16a95-d13c-494d-88f9-b7bab2ffddbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:02.713181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b16a95-d13c-494d-88f9-b7bab2ffddbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:02.714830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b16a95-d13c-494d-88f9-b7bab2ffddbc None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:02.723895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 186/389] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:02.730798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b16a95-d13c-494d-88f9-b7bab2ffddbc None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:02.816124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:02.841883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b16a95-d13c-494d-88f9-b7bab2ffddbc None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:02.879937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a96e6-c051-4c2b-95c8-cc953ded0562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:02.880683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a96e6-c051-4c2b-95c8-cc953ded0562 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:02.880965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a96e6-c051-4c2b-95c8-cc953ded0562 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:02.881241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a96e6-c051-4c2b-95c8-cc953ded0562 None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:02.882467 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a96e6-c051-4c2b-95c8-cc953ded0562 None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:02.883912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a96e6-c051-4c2b-95c8-cc953ded0562 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:02.885846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 187/390] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:02 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:02.891963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a96e6-c051-4c2b-95c8-cc953ded0562 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:02.896097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0ecb4-9476-4a61-80b8-0cfaeff05273 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:02.925080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:02.983319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be1b3b-784f-4419-926e-74f4cdfbddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:02.984605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be1b3b-784f-4419-926e-74f4cdfbddbb None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:02.984605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be1b3b-784f-4419-926e-74f4cdfbddbb None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:02.984605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be1b3b-784f-4419-926e-74f4cdfbddbb None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:02.985790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be1b3b-784f-4419-926e-74f4cdfbddbb None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:02.987256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be1b3b-784f-4419-926e-74f4cdfbddbb None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:02.988888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 204/391] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:02 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:02.995294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be1b3b-784f-4419-926e-74f4cdfbddbb None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:03.019758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a96e6-c051-4c2b-95c8-cc953ded0562 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:03.059467 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729489-314c-460e-ae41-4be1b149f0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:03.060505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729489-314c-460e-ae41-4be1b149f0a0 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:03.061036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729489-314c-460e-ae41-4be1b149f0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:03.061498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e729489-314c-460e-ae41-4be1b149f0a0 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:03.062986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729489-314c-460e-ae41-4be1b149f0a0 None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:03.064756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e729489-314c-460e-ae41-4be1b149f0a0 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:03.066940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 188/392] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:02 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:03.073255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729489-314c-460e-ae41-4be1b149f0a0 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:03.090309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:03.215530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be1b3b-784f-4419-926e-74f4cdfbddbb None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:03.259276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e729489-314c-460e-ae41-4be1b149f0a0 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:03.301325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc0aa0-a11b-4670-940e-8f0ce93b09da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:03.302518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc0aa0-a11b-4670-940e-8f0ce93b09da None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:03.303017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc0aa0-a11b-4670-940e-8f0ce93b09da None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:03.303480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc0aa0-a11b-4670-940e-8f0ce93b09da None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1/groups/9908b486ccf84e6ea374363f9ae8c5e5/roles/20d35b10934447938237c0db5202bbfb` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:03.305040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0c4bb-715a-43d6-8a6e-1ee9c7a21ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:03.305810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0c4bb-715a-43d6-8a6e-1ee9c7a21ff2 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:03.306118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0c4bb-715a-43d6-8a6e-1ee9c7a21ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:03.306390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0c4bb-715a-43d6-8a6e-1ee9c7a21ff2 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1/groups/9908b486ccf84e6ea374363f9ae8c5e5/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:03.306846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc0aa0-a11b-4670-940e-8f0ce93b09da None admin] RBAC: Authorizing `identity:create_grant(project_id=28843c17667d4709ab63a1378bc693a1, group_id=9908b486ccf84e6ea374363f9ae8c5e5, role_id=20d35b10934447938237c0db5202bbfb)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:03.308635 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc0aa0-a11b-4670-940e-8f0ce93b09da None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:03.315050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0c4bb-715a-43d6-8a6e-1ee9c7a21ff2 None admin] RBAC: Authorizing `identity:create_grant(project_id=28843c17667d4709ab63a1378bc693a1, group_id=9908b486ccf84e6ea374363f9ae8c5e5, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:03.317004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0c4bb-715a-43d6-8a6e-1ee9c7a21ff2 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:03.374057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 189/393] 10.222.0.79 () {58 vars in 1442 bytes} [Tue Aug 9 16:32:03 2022] PUT /identity/v3/projects/28843c17667d4709ab63a1378bc693a1/groups/9908b486ccf84e6ea374363f9ae8c5e5/roles/20d35b10934447938237c0db5202bbfb => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:03.413718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 205/394] 10.222.0.79 () {58 vars in 1442 bytes} [Tue Aug 9 16:32:02 2022] PUT /identity/v3/projects/28843c17667d4709ab63a1378bc693a1/groups/9908b486ccf84e6ea374363f9ae8c5e5/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:03.427842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fa00b-5c1a-47b7-97b9-cf0974143592 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:03.428491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fa00b-5c1a-47b7-97b9-cf0974143592 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:03.429036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fa00b-5c1a-47b7-97b9-cf0974143592 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:03.430769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 190/395] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:03.437127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d772dd-ca70-4976-beb9-1a677f22f277 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:03.437913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d772dd-ca70-4976-beb9-1a677f22f277 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:03.438431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d772dd-ca70-4976-beb9-1a677f22f277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:03.459725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-58d772dd-ca70-4976-beb9-1a677f22f277 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:03.461353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-58d772dd-ca70-4976-beb9-1a677f22f277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:03.504564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:03.548947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 206/396] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:03.567454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fa00b-5c1a-47b7-97b9-cf0974143592 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:03.652707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:03.699377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fa00b-5c1a-47b7-97b9-cf0974143592 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:03.735331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1214ff-8e04-41d4-a801-6fcd0e79962f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:03.736384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1214ff-8e04-41d4-a801-6fcd0e79962f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:03.736851 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1214ff-8e04-41d4-a801-6fcd0e79962f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:03.737322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1214ff-8e04-41d4-a801-6fcd0e79962f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:03.750307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1214ff-8e04-41d4-a801-6fcd0e79962f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-bc1214ff-8e04-41d4-a801-6fcd0e79962f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:03.757416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:03.758330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:32:03.759139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:03.761772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 191/397] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:03.767829 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d772dd-ca70-4976-beb9-1a677f22f277 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:03.886764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d772dd-ca70-4976-beb9-1a677f22f277 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:03.931521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a1f7e-540b-4611-b4de-b79e9cfa4526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:03.932138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a1f7e-540b-4611-b4de-b79e9cfa4526 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:03.932414 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a1f7e-540b-4611-b4de-b79e9cfa4526 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:03.932681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a1f7e-540b-4611-b4de-b79e9cfa4526 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:03.933402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a1f7e-540b-4611-b4de-b79e9cfa4526 None admin] RBAC: Adding query filter params (name=member) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:03.933705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a1f7e-540b-4611-b4de-b79e9cfa4526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:03.935089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a1f7e-540b-4611-b4de-b79e9cfa4526 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:03.948465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 207/398] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:32:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:03.954127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1214ff-8e04-41d4-a801-6fcd0e79962f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:04.103177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1214ff-8e04-41d4-a801-6fcd0e79962f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:04.138658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:04.150949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4201d2b-be90-4d0b-aa0d-56bffec756c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:04.151560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4201d2b-be90-4d0b-aa0d-56bffec756c4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:04.151835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4201d2b-be90-4d0b-aa0d-56bffec756c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:04.152095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4201d2b-be90-4d0b-aa0d-56bffec756c4 None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:04.158696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4201d2b-be90-4d0b-aa0d-56bffec756c4 None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:04.160231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4201d2b-be90-4d0b-aa0d-56bffec756c4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:04.162390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 192/399] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:03 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:04.168059 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a1f7e-540b-4611-b4de-b79e9cfa4526 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:04.210306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:04.289296 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a1f7e-540b-4611-b4de-b79e9cfa4526 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:04.329576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62be240-9118-4ca2-a187-c8b4af7a7dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:04.330242 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62be240-9118-4ca2-a187-c8b4af7a7dd9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:04.330579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62be240-9118-4ca2-a187-c8b4af7a7dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:04.330935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62be240-9118-4ca2-a187-c8b4af7a7dd9 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:04.332110 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62be240-9118-4ca2-a187-c8b4af7a7dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:04.333561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62be240-9118-4ca2-a187-c8b4af7a7dd9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:04.335258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 208/400] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:04 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:04.341355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4201d2b-be90-4d0b-aa0d-56bffec756c4 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:04.403621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9070f-4645-4328-aefa-9eb184b373da None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:04.403901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9070f-4645-4328-aefa-9eb184b373da None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:04.404156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9070f-4645-4328-aefa-9eb184b373da None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:04.405401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 209/401] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:04.410906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c3ba8-5033-4423-987d-8e73b9843d87 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:04.411449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c3ba8-5033-4423-987d-8e73b9843d87 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:04.412491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c3ba8-5033-4423-987d-8e73b9843d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:04.447601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-7b2c3ba8-5033-4423-987d-8e73b9843d87 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:04.449275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7b2c3ba8-5033-4423-987d-8e73b9843d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:04.468154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4201d2b-be90-4d0b-aa0d-56bffec756c4 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:04.540536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d21dd1-451f-4238-90d8-71a32fd63d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:04.541204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d21dd1-451f-4238-90d8-71a32fd63d99 None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:04.541475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d21dd1-451f-4238-90d8-71a32fd63d99 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:04.541742 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d21dd1-451f-4238-90d8-71a32fd63d99 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689/groups/9908b486ccf84e6ea374363f9ae8c5e5/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:04.544630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d21dd1-451f-4238-90d8-71a32fd63d99 None admin] RBAC: Authorizing `identity:create_grant(project_id=5796807ec0ad4cab845934a9df6b9689, group_id=9908b486ccf84e6ea374363f9ae8c5e5, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:04.546078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d21dd1-451f-4238-90d8-71a32fd63d99 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:04.573103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 210/402] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:04.596793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c3ba8-5033-4423-987d-8e73b9843d87 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:04.629548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 193/403] 10.222.0.79 () {58 vars in 1442 bytes} [Tue Aug 9 16:32:04 2022] PUT /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689/groups/9908b486ccf84e6ea374363f9ae8c5e5/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:04.638527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:04.653784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe05db-c76a-420d-9e64-2b6bd27e66ff None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:04.653988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe05db-c76a-420d-9e64-2b6bd27e66ff None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:04.654269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe05db-c76a-420d-9e64-2b6bd27e66ff None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:04.655739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 194/404] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:04.666454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f0881-c6dc-4621-a15e-4b778efa2e20 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:04.666814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f0881-c6dc-4621-a15e-4b778efa2e20 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:04.667161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f0881-c6dc-4621-a15e-4b778efa2e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:04.706373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-736f0881-c6dc-4621-a15e-4b778efa2e20 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:04.707594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-736f0881-c6dc-4621-a15e-4b778efa2e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:04.727975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c3ba8-5033-4423-987d-8e73b9843d87 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:04.797828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d4ebf-4d06-4ad9-8102-6be302dc9648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:04.798517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d4ebf-4d06-4ad9-8102-6be302dc9648 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:04.798794 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d4ebf-4d06-4ad9-8102-6be302dc9648 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:04.799066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d4ebf-4d06-4ad9-8102-6be302dc9648 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:04.812201 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:04.826492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d4ebf-4d06-4ad9-8102-6be302dc9648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-f85d4ebf-4d06-4ad9-8102-6be302dc9648 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:04.832668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:04.833624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:04.835058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 211/405] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:04.843901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d4ebf-4d06-4ad9-8102-6be302dc9648 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:04.856969 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 195/406] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:04.880014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f0881-c6dc-4621-a15e-4b778efa2e20 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:04.965656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d4ebf-4d06-4ad9-8102-6be302dc9648 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:04.978668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:05.014554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf34bc-3289-4c88-bbf0-6681781c8ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:05.015160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf34bc-3289-4c88-bbf0-6681781c8ece None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:05.015436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf34bc-3289-4c88-bbf0-6681781c8ece None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:05.015705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf34bc-3289-4c88-bbf0-6681781c8ece None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:05.016368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf34bc-3289-4c88-bbf0-6681781c8ece None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:05.016665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf34bc-3289-4c88-bbf0-6681781c8ece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:05.018068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf34bc-3289-4c88-bbf0-6681781c8ece None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:05.034119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 212/407] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:05.047050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf34bc-3289-4c88-bbf0-6681781c8ece None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:05.051983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f0881-c6dc-4621-a15e-4b778efa2e20 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:05.101710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18b145-2fac-4c44-861c-91d0c3ee6841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:05.103449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18b145-2fac-4c44-861c-91d0c3ee6841 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:05.103732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18b145-2fac-4c44-861c-91d0c3ee6841 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:05.103998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18b145-2fac-4c44-861c-91d0c3ee6841 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:05.133984 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f18b145-2fac-4c44-861c-91d0c3ee6841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-4f18b145-2fac-4c44-861c-91d0c3ee6841 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:05.142056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:05.144396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:05.146364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 196/408] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:05.153974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18b145-2fac-4c44-861c-91d0c3ee6841 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:05.212595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:05.272690 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf34bc-3289-4c88-bbf0-6681781c8ece None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:05.282695 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18b145-2fac-4c44-861c-91d0c3ee6841 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:05.323319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2c1c-012f-48f0-a5bc-b78d5549e03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:05.323921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2c1c-012f-48f0-a5bc-b78d5549e03e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:05.324837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2c1c-012f-48f0-a5bc-b78d5549e03e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:05.325157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a2c1c-012f-48f0-a5bc-b78d5549e03e None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:05.325942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a2c1c-012f-48f0-a5bc-b78d5549e03e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:05.326266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a2c1c-012f-48f0-a5bc-b78d5549e03e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:05.327635 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a2c1c-012f-48f0-a5bc-b78d5549e03e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:05.338379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 197/409] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:05.346946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2c1c-012f-48f0-a5bc-b78d5549e03e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:05.358517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc622f-a264-4f20-b06e-f0cb1c902ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:05.359153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc622f-a264-4f20-b06e-f0cb1c902ca8 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:05.359399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc622f-a264-4f20-b06e-f0cb1c902ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:05.359666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc622f-a264-4f20-b06e-f0cb1c902ca8 None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:05.360780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc622f-a264-4f20-b06e-f0cb1c902ca8 None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:05.362258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc622f-a264-4f20-b06e-f0cb1c902ca8 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:05.363812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 213/410] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:05 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:05.369279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc622f-a264-4f20-b06e-f0cb1c902ca8 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:05.399882 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:05.519238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a2c1c-012f-48f0-a5bc-b78d5549e03e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:05.528313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:05.559001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc622f-a264-4f20-b06e-f0cb1c902ca8 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:05.580480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2666d8c-733d-4b12-bbf9-0273f3068684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:05.581104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666d8c-733d-4b12-bbf9-0273f3068684 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:05.581377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666d8c-733d-4b12-bbf9-0273f3068684 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:05.581644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666d8c-733d-4b12-bbf9-0273f3068684 None admin] PATH_INFO: `/v3/groups/ae7fd0887fc946ec8845d5b128694719` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:05.583617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2666d8c-733d-4b12-bbf9-0273f3068684 None admin] RBAC: Authorizing `identity:get_group(group_id=ae7fd0887fc946ec8845d5b128694719)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:05.585231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2666d8c-733d-4b12-bbf9-0273f3068684 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:05.588168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 198/411] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:05 2022] GET /identity/v3/groups/ae7fd0887fc946ec8845d5b128694719 => generated 229 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:05.595431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2666d8c-733d-4b12-bbf9-0273f3068684 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:05.636801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f614b-6684-4af9-bfea-8e543e6cd242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:05.637426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f614b-6684-4af9-bfea-8e543e6cd242 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:05.637704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f614b-6684-4af9-bfea-8e543e6cd242 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:05.637989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f614b-6684-4af9-bfea-8e543e6cd242 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:05.639794 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f614b-6684-4af9-bfea-8e543e6cd242 None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:05.641239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f614b-6684-4af9-bfea-8e543e6cd242 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:05.644012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 214/412] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:05 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:05.650995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f614b-6684-4af9-bfea-8e543e6cd242 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:05.782387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:05.801033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f614b-6684-4af9-bfea-8e543e6cd242 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:05.806357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2666d8c-733d-4b12-bbf9-0273f3068684 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:05.847146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967cdbf0-f670-42dd-9161-fa0fab7e5116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:05.847832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cdbf0-f670-42dd-9161-fa0fab7e5116 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:05.848174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cdbf0-f670-42dd-9161-fa0fab7e5116 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:05.848453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cdbf0-f670-42dd-9161-fa0fab7e5116 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689/groups/9908b486ccf84e6ea374363f9ae8c5e5/roles/20d35b10934447938237c0db5202bbfb` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:05.848754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba25c0-1079-4a5b-972b-cb0d9b2fbf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:05.850127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba25c0-1079-4a5b-972b-cb0d9b2fbf89 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:05.850607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba25c0-1079-4a5b-972b-cb0d9b2fbf89 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:05.851071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba25c0-1079-4a5b-972b-cb0d9b2fbf89 None admin] PATH_INFO: `/v3/projects/e0a53b8ab06c430ea7305247e9df6f92` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:05.851171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967cdbf0-f670-42dd-9161-fa0fab7e5116 None admin] RBAC: Authorizing `identity:create_grant(project_id=5796807ec0ad4cab845934a9df6b9689, group_id=9908b486ccf84e6ea374363f9ae8c5e5, role_id=20d35b10934447938237c0db5202bbfb)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:05.852488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba25c0-1079-4a5b-972b-cb0d9b2fbf89 None admin] RBAC: Authorizing `identity:get_project(project_id=e0a53b8ab06c430ea7305247e9df6f92)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:05.852592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967cdbf0-f670-42dd-9161-fa0fab7e5116 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:05.854230 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba25c0-1079-4a5b-972b-cb0d9b2fbf89 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:05.857167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 199/413] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:05 2022] GET /identity/v3/projects/e0a53b8ab06c430ea7305247e9df6f92 => generated 344 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:05.858783 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68643d63-a6b4-4f3a-b89c-e5686bf78c96 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:05.859264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68643d63-a6b4-4f3a-b89c-e5686bf78c96 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:05.859718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68643d63-a6b4-4f3a-b89c-e5686bf78c96 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:05.865138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 200/414] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:05.866589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68643d63-a6b4-4f3a-b89c-e5686bf78c96 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:05.915689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:05.946207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 215/415] 10.222.0.79 () {58 vars in 1442 bytes} [Tue Aug 9 16:32:05 2022] PUT /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689/groups/9908b486ccf84e6ea374363f9ae8c5e5/roles/20d35b10934447938237c0db5202bbfb => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:05.947987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18beb7-902e-4950-ba40-6cf0ce2dc5a9 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:05.948475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18beb7-902e-4950-ba40-6cf0ce2dc5a9 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:05.948946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18beb7-902e-4950-ba40-6cf0ce2dc5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:05.980372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9a18beb7-902e-4950-ba40-6cf0ce2dc5a9 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:05.980791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68643d63-a6b4-4f3a-b89c-e5686bf78c96 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:05.982576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9a18beb7-902e-4950-ba40-6cf0ce2dc5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:06.045726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc296b-4896-4e9c-85e5-20eaa45c655d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:06.046344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc296b-4896-4e9c-85e5-20eaa45c655d None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:06.046619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc296b-4896-4e9c-85e5-20eaa45c655d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:06.046890 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc296b-4896-4e9c-85e5-20eaa45c655d None admin] PATH_INFO: `/v3/projects/e0a53b8ab06c430ea7305247e9df6f92/groups/ae7fd0887fc946ec8845d5b128694719/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:06.053107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc296b-4896-4e9c-85e5-20eaa45c655d None admin] RBAC: Authorizing `identity:create_grant(project_id=e0a53b8ab06c430ea7305247e9df6f92, group_id=ae7fd0887fc946ec8845d5b128694719, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:06.054718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc296b-4896-4e9c-85e5-20eaa45c655d None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:06.074079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 216/416] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:06.097740 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18beb7-902e-4950-ba40-6cf0ce2dc5a9 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:06.151998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 201/417] 10.222.0.79 () {58 vars in 1442 bytes} [Tue Aug 9 16:32:05 2022] PUT /identity/v3/projects/e0a53b8ab06c430ea7305247e9df6f92/groups/ae7fd0887fc946ec8845d5b128694719/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:06.198146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:06.225942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18beb7-902e-4950-ba40-6cf0ce2dc5a9 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:06.266574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fe90c-e3e2-4d50-a7c2-16df787a7a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:06.267219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fe90c-e3e2-4d50-a7c2-16df787a7a8f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:06.267495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fe90c-e3e2-4d50-a7c2-16df787a7a8f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:06.267764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fe90c-e3e2-4d50-a7c2-16df787a7a8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:06.280974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fe90c-e3e2-4d50-a7c2-16df787a7a8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-a45fe90c-e3e2-4d50-a7c2-16df787a7a8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:06.286115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:06.287614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:32:06.288419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:06.289021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 217/418] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:06.299195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc296b-4896-4e9c-85e5-20eaa45c655d None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:06.299736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982e532-7ceb-4c8e-b153-f4a30f44417f None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:06.300028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982e532-7ceb-4c8e-b153-f4a30f44417f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:06.300305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982e532-7ceb-4c8e-b153-f4a30f44417f None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:06.301666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 218/419] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:06.309201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eb013-ad7a-4141-9c21-f9bd97bb46f8 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:06.309516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eb013-ad7a-4141-9c21-f9bd97bb46f8 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:06.309805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2eb013-ad7a-4141-9c21-f9bd97bb46f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:06.333632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-6c2eb013-ad7a-4141-9c21-f9bd97bb46f8 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:06.335444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6c2eb013-ad7a-4141-9c21-f9bd97bb46f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:06.422339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 219/420] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:32:06.452896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2eb013-ad7a-4141-9c21-f9bd97bb46f8 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:06.473002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc296b-4896-4e9c-85e5-20eaa45c655d None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:06.475269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:06.545515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202fdf0-6929-4457-9197-f8dce649d97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:06.546139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202fdf0-6929-4457-9197-f8dce649d97c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:06.546410 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202fdf0-6929-4457-9197-f8dce649d97c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:06.546670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b202fdf0-6929-4457-9197-f8dce649d97c None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:06.547328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202fdf0-6929-4457-9197-f8dce649d97c None admin] RBAC: Adding query filter params (name=member) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:06.547619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202fdf0-6929-4457-9197-f8dce649d97c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:06.551239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:06.553944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b202fdf0-6929-4457-9197-f8dce649d97c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:06.570396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 202/421] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:32:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:06.576230 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202fdf0-6929-4457-9197-f8dce649d97c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:06.621719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2eb013-ad7a-4141-9c21-f9bd97bb46f8 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:06.710005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618252de-4654-443a-ae82-82c4f69739d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:06.710654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618252de-4654-443a-ae82-82c4f69739d6 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:06.710923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618252de-4654-443a-ae82-82c4f69739d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:06.711185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618252de-4654-443a-ae82-82c4f69739d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:06.718060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b202fdf0-6929-4457-9197-f8dce649d97c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:06.744718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618252de-4654-443a-ae82-82c4f69739d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-618252de-4654-443a-ae82-82c4f69739d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:06.750009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:06.750885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:06.751677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 220/422] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:06.759520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab536ae4-ffe2-4925-95a6-239008f1384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:06.760167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618252de-4654-443a-ae82-82c4f69739d6 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:06.760274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab536ae4-ffe2-4925-95a6-239008f1384a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:06.760479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab536ae4-ffe2-4925-95a6-239008f1384a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:06.760840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab536ae4-ffe2-4925-95a6-239008f1384a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:06.802540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:06.838664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab536ae4-ffe2-4925-95a6-239008f1384a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:06.840389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab536ae4-ffe2-4925-95a6-239008f1384a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:06.916643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618252de-4654-443a-ae82-82c4f69739d6 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:06.943620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:06.951308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca521a5-3e5b-47fd-873d-17a6c248b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:06.951986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca521a5-3e5b-47fd-873d-17a6c248b108 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:06.952328 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca521a5-3e5b-47fd-873d-17a6c248b108 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:06.952708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca521a5-3e5b-47fd-873d-17a6c248b108 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:06.953424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca521a5-3e5b-47fd-873d-17a6c248b108 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:06.953721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca521a5-3e5b-47fd-873d-17a6c248b108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:06.955121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca521a5-3e5b-47fd-873d-17a6c248b108 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:06.958689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 203/423] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:06.959598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbd31f-9938-4332-abed-dbec7cc15bf5 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:06.959876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbd31f-9938-4332-abed-dbec7cc15bf5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:06.960142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbd31f-9938-4332-abed-dbec7cc15bf5 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:06.966274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 221/425] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:06.966612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 204/425] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:06.968330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244c633-8f70-45f7-bf0c-0d9afe511464 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:06.968607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244c633-8f70-45f7-bf0c-0d9afe511464 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:06.968870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3244c633-8f70-45f7-bf0c-0d9afe511464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:06.970606 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca521a5-3e5b-47fd-873d-17a6c248b108 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:06.989505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-3244c633-8f70-45f7-bf0c-0d9afe511464 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:06.990779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3244c633-8f70-45f7-bf0c-0d9afe511464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:07.062479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:07.070068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 205/426] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:32:07.070981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244c633-8f70-45f7-bf0c-0d9afe511464 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:07.076971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca521a5-3e5b-47fd-873d-17a6c248b108 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:07.112474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb20b9-e9d9-4e8a-ab84-b152d0ba9e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:07.113201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb20b9-e9d9-4e8a-ab84-b152d0ba9e84 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:07.113541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb20b9-e9d9-4e8a-ab84-b152d0ba9e84 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:07.113873 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb20b9-e9d9-4e8a-ab84-b152d0ba9e84 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:07.115119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb20b9-e9d9-4e8a-ab84-b152d0ba9e84 None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:07.116664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb20b9-e9d9-4e8a-ab84-b152d0ba9e84 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:07.118371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 222/427] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:06 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:07.119245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb20b9-e9d9-4e8a-ab84-b152d0ba9e84 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:07.180537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3244c633-8f70-45f7-bf0c-0d9afe511464 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:07.182334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:07.220366 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60db3d6-dceb-42a7-8a2d-02245bea6492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:07.220981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60db3d6-dceb-42a7-8a2d-02245bea6492 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:07.221258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60db3d6-dceb-42a7-8a2d-02245bea6492 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:07.221531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60db3d6-dceb-42a7-8a2d-02245bea6492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:07.234642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb20b9-e9d9-4e8a-ab84-b152d0ba9e84 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:07.270461 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48be9c3-ebc1-4e5c-af62-37745869031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:07.271316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48be9c3-ebc1-4e5c-af62-37745869031d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:07.272243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48be9c3-ebc1-4e5c-af62-37745869031d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:07.272718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48be9c3-ebc1-4e5c-af62-37745869031d None admin] PATH_INFO: `/v3/roles/member` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:07.284193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48be9c3-ebc1-4e5c-af62-37745869031d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-b48be9c3-ebc1-4e5c-af62-37745869031d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:07.289286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:07.290230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:07.291035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 223/428] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:07.297979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48be9c3-ebc1-4e5c-af62-37745869031d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:07.298709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:07.307919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60db3d6-dceb-42a7-8a2d-02245bea6492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:07.310767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60db3d6-dceb-42a7-8a2d-02245bea6492 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:07.400988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48be9c3-ebc1-4e5c-af62-37745869031d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:07.426349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:07.444755 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b824a-0cbe-4067-88bd-9f2f7a7d9a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:07.445382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b824a-0cbe-4067-88bd-9f2f7a7d9a7c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:07.445659 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b824a-0cbe-4067-88bd-9f2f7a7d9a7c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:07.445929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b824a-0cbe-4067-88bd-9f2f7a7d9a7c None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:07.447080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b824a-0cbe-4067-88bd-9f2f7a7d9a7c None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:07.448681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b824a-0cbe-4067-88bd-9f2f7a7d9a7c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:07.450620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 224/429] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:07 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:07.451625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b824a-0cbe-4067-88bd-9f2f7a7d9a7c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:07.459765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 206/430] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:07.460947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60db3d6-dceb-42a7-8a2d-02245bea6492 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:07.579894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:07.583556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b824a-0cbe-4067-88bd-9f2f7a7d9a7c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:07.589517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60db3d6-dceb-42a7-8a2d-02245bea6492 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:07.621380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed1806-bc47-4c22-a8f2-39840b57eb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:07.622356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed1806-bc47-4c22-a8f2-39840b57eb08 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:07.622797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed1806-bc47-4c22-a8f2-39840b57eb08 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:07.625371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed1806-bc47-4c22-a8f2-39840b57eb08 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:07.625371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed1806-bc47-4c22-a8f2-39840b57eb08 None admin] RBAC: Adding query filter params (name=member) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:07.625371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed1806-bc47-4c22-a8f2-39840b57eb08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:07.626302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed1806-bc47-4c22-a8f2-39840b57eb08 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:07.635326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00f678-0ea3-4c0c-8d65-782e8b6ec991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:07.635598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 225/431] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:32:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:07.636415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed1806-bc47-4c22-a8f2-39840b57eb08 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:07.636541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00f678-0ea3-4c0c-8d65-782e8b6ec991 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:07.636719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00f678-0ea3-4c0c-8d65-782e8b6ec991 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:07.637136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00f678-0ea3-4c0c-8d65-782e8b6ec991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:07.638105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00f678-0ea3-4c0c-8d65-782e8b6ec991 None admin] RBAC: Adding query filter params (group.id=9908b486ccf84e6ea374363f9ae8c5e5, role.id=2490390ea61f4cf594df3587e7674650, scope.project.id=28843c17667d4709ab63a1378bc693a1) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:07.638564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00f678-0ea3-4c0c-8d65-782e8b6ec991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:07.640542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00f678-0ea3-4c0c-8d65-782e8b6ec991 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:07.658683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 207/432] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Aug 9 16:32:07 2022] GET /identity/v3/role_assignments?group.id=9908b486ccf84e6ea374363f9ae8c5e5&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=2490390ea61f4cf594df3587e7674650 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:07.658683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00f678-0ea3-4c0c-8d65-782e8b6ec991 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:07.764758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:07.790258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00f678-0ea3-4c0c-8d65-782e8b6ec991 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:07.832345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe2d30-f179-4c10-9213-c94ec98d397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:07.833320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe2d30-f179-4c10-9213-c94ec98d397c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:07.833829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe2d30-f179-4c10-9213-c94ec98d397c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:07.835319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe2d30-f179-4c10-9213-c94ec98d397c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:07.869494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:07.893094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed1806-bc47-4c22-a8f2-39840b57eb08 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:07.922557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe2d30-f179-4c10-9213-c94ec98d397c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:07.923928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe2d30-f179-4c10-9213-c94ec98d397c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:07.941770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8fe7-5c1e-4cdc-99be-d139732ff744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:07.942608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8fe7-5c1e-4cdc-99be-d139732ff744 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:07.942892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8fe7-5c1e-4cdc-99be-d139732ff744 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:07.943167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8fe7-5c1e-4cdc-99be-d139732ff744 None admin] PATH_INFO: `/v3/projects/28843c17667d4709ab63a1378bc693a1` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:07.944905 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8fe7-5c1e-4cdc-99be-d139732ff744 None admin] RBAC: Authorizing `identity:get_project(project_id=28843c17667d4709ab63a1378bc693a1)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:07.946942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8fe7-5c1e-4cdc-99be-d139732ff744 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:07.949357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 226/433] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:07 2022] GET /identity/v3/projects/28843c17667d4709ab63a1378bc693a1 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:07.955216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8fe7-5c1e-4cdc-99be-d139732ff744 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:08.043991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 208/434] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:08.048931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe2d30-f179-4c10-9213-c94ec98d397c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:08.078343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8fe7-5c1e-4cdc-99be-d139732ff744 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:08.095670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:08.117012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bff9a-060f-4b0f-aeb0-4d89ff48c03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:08.117647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bff9a-060f-4b0f-aeb0-4d89ff48c03c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.117924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bff9a-060f-4b0f-aeb0-4d89ff48c03c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.118224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02bff9a-060f-4b0f-aeb0-4d89ff48c03c None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.118577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:08.119555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bff9a-060f-4b0f-aeb0-4d89ff48c03c None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:08.121735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02bff9a-060f-4b0f-aeb0-4d89ff48c03c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:08.124252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 227/435] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:07 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:08.130349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bff9a-060f-4b0f-aeb0-4d89ff48c03c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:08.171334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe2d30-f179-4c10-9213-c94ec98d397c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:08.216120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d3ba5-2286-4eec-814d-e47999b1faf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:08.216680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d3ba5-2286-4eec-814d-e47999b1faf1 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.216960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d3ba5-2286-4eec-814d-e47999b1faf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.217228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d3ba5-2286-4eec-814d-e47999b1faf1 None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.218409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d3ba5-2286-4eec-814d-e47999b1faf1 None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:08.219832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d3ba5-2286-4eec-814d-e47999b1faf1 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:08.225808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 209/436] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:08 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:08.227041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d3ba5-2286-4eec-814d-e47999b1faf1 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:08.254780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02bff9a-060f-4b0f-aeb0-4d89ff48c03c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:08.297938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b1389-a07e-447a-80ec-9ceaca117332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:08.298959 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b1389-a07e-447a-80ec-9ceaca117332 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.299526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b1389-a07e-447a-80ec-9ceaca117332 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.299992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b1389-a07e-447a-80ec-9ceaca117332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.300958 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b1389-a07e-447a-80ec-9ceaca117332 None admin] RBAC: Adding query filter params (group.id=9908b486ccf84e6ea374363f9ae8c5e5, role.id=20d35b10934447938237c0db5202bbfb, scope.project.id=28843c17667d4709ab63a1378bc693a1) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:08.301478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b1389-a07e-447a-80ec-9ceaca117332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:08.303948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b1389-a07e-447a-80ec-9ceaca117332 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:08.313109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 228/437] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Aug 9 16:32:08 2022] GET /identity/v3/role_assignments?group.id=9908b486ccf84e6ea374363f9ae8c5e5&scope.project.id=28843c17667d4709ab63a1378bc693a1&role.id=20d35b10934447938237c0db5202bbfb => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:08.355440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d3ba5-2286-4eec-814d-e47999b1faf1 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:08.375323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd22f5-2eb5-4dc4-acb0-45ec2b8d5107 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.376040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd22f5-2eb5-4dc4-acb0-45ec2b8d5107 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.376572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd22f5-2eb5-4dc4-acb0-45ec2b8d5107 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.378148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 229/438] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:08.384777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc73e27-726b-4c4e-b767-bf76aacd26d0 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.385070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc73e27-726b-4c4e-b767-bf76aacd26d0 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.385325 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc73e27-726b-4c4e-b767-bf76aacd26d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.401052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:08.410252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-4bc73e27-726b-4c4e-b767-bf76aacd26d0 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:08.410252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4bc73e27-726b-4c4e-b767-bf76aacd26d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:08.424861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b9ad4-666b-4cc0-9e7b-7312d6d3a4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:08.426244 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b9ad4-666b-4cc0-9e7b-7312d6d3a4fb None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.426745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b9ad4-666b-4cc0-9e7b-7312d6d3a4fb None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.427502 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b9ad4-666b-4cc0-9e7b-7312d6d3a4fb None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.430700 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b9ad4-666b-4cc0-9e7b-7312d6d3a4fb None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:08.432973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b9ad4-666b-4cc0-9e7b-7312d6d3a4fb None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:08.435992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 210/439] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:08 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:08.444288 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b9ad4-666b-4cc0-9e7b-7312d6d3a4fb None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:08.473292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:08.512101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 230/440] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:32:08.513488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712abe86-ab64-4af1-b5a5-82bdb4ce46d8 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.513937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712abe86-ab64-4af1-b5a5-82bdb4ce46d8 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.514498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712abe86-ab64-4af1-b5a5-82bdb4ce46d8 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.516897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 231/441] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:08.522443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36d4b3-6a25-4f23-b159-4e0065cc2e60 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.522904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36d4b3-6a25-4f23-b159-4e0065cc2e60 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.523324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36d4b3-6a25-4f23-b159-4e0065cc2e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.543790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-7f36d4b3-6a25-4f23-b159-4e0065cc2e60 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:08.545320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7f36d4b3-6a25-4f23-b159-4e0065cc2e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:08.590273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b9ad4-666b-4cc0-9e7b-7312d6d3a4fb None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:08.633772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71415cec-152a-4418-99ca-a7f35861945f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:08.634418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71415cec-152a-4418-99ca-a7f35861945f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.634859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71415cec-152a-4418-99ca-a7f35861945f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.635273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71415cec-152a-4418-99ca-a7f35861945f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.636139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71415cec-152a-4418-99ca-a7f35861945f None admin] RBAC: Adding query filter params (group.id=9908b486ccf84e6ea374363f9ae8c5e5, role.id=2490390ea61f4cf594df3587e7674650, scope.project.id=5796807ec0ad4cab845934a9df6b9689) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:08.636588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71415cec-152a-4418-99ca-a7f35861945f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:08.637669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 232/442] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:32:08.638381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71415cec-152a-4418-99ca-a7f35861945f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:08.642779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36d4b3-6a25-4f23-b159-4e0065cc2e60 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:08.645056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 211/443] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Aug 9 16:32:08 2022] GET /identity/v3/role_assignments?group.id=9908b486ccf84e6ea374363f9ae8c5e5&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=2490390ea61f4cf594df3587e7674650 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:08.661222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71415cec-152a-4418-99ca-a7f35861945f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:08.743482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:08.781280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36d4b3-6a25-4f23-b159-4e0065cc2e60 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:08.792354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:08.798541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71415cec-152a-4418-99ca-a7f35861945f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:08.832050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457c540-59f7-4358-b76c-69f4d2b14fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:08.832692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457c540-59f7-4358-b76c-69f4d2b14fe3 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.832971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457c540-59f7-4358-b76c-69f4d2b14fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.833245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457c540-59f7-4358-b76c-69f4d2b14fe3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.839559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde65e71-54cc-44a4-b1eb-7bdf1df080f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:08.840476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65e71-54cc-44a4-b1eb-7bdf1df080f8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:08.840810 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65e71-54cc-44a4-b1eb-7bdf1df080f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:08.841115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde65e71-54cc-44a4-b1eb-7bdf1df080f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:08.847341 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457c540-59f7-4358-b76c-69f4d2b14fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-8457c540-59f7-4358-b76c-69f4d2b14fe3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:08.852558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:08.853462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:08.854369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 233/444] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:08.855199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde65e71-54cc-44a4-b1eb-7bdf1df080f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-fde65e71-54cc-44a4-b1eb-7bdf1df080f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:08.859925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:08.860829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:08.861627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:08.862298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457c540-59f7-4358-b76c-69f4d2b14fe3 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:08.862368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 212/445] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:08.867908 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde65e71-54cc-44a4-b1eb-7bdf1df080f8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:08.981892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457c540-59f7-4358-b76c-69f4d2b14fe3 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:08.992269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde65e71-54cc-44a4-b1eb-7bdf1df080f8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:09.015154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:09.026061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c614fd4-cd7b-4859-9c96-3a51c1f5dd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:09.033871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c614fd4-cd7b-4859-9c96-3a51c1f5dd4f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:09.033871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c614fd4-cd7b-4859-9c96-3a51c1f5dd4f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:09.033871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c614fd4-cd7b-4859-9c96-3a51c1f5dd4f None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:09.033871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c614fd4-cd7b-4859-9c96-3a51c1f5dd4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:09.033871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c614fd4-cd7b-4859-9c96-3a51c1f5dd4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:09.033871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c614fd4-cd7b-4859-9c96-3a51c1f5dd4f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:09.036889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d07847-da31-42fd-a31d-783f16f048e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:09.038078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d07847-da31-42fd-a31d-783f16f048e7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:09.038340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d07847-da31-42fd-a31d-783f16f048e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:09.038553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 234/446] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:09.038815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d07847-da31-42fd-a31d-783f16f048e7 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:09.039259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d07847-da31-42fd-a31d-783f16f048e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:09.039565 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d07847-da31-42fd-a31d-783f16f048e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:09.040921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d07847-da31-42fd-a31d-783f16f048e7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:09.044010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c614fd4-cd7b-4859-9c96-3a51c1f5dd4f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:09.049974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 213/447] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:09.055073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d07847-da31-42fd-a31d-783f16f048e7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:09.074692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:09.141926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c614fd4-cd7b-4859-9c96-3a51c1f5dd4f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:09.155399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d07847-da31-42fd-a31d-783f16f048e7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:09.174149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a96466-5c95-4847-991b-1398b93eef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:09.174733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a96466-5c95-4847-991b-1398b93eef8e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:09.175009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a96466-5c95-4847-991b-1398b93eef8e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:09.175268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a96466-5c95-4847-991b-1398b93eef8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:09.188261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe92c1d-5ad1-48fe-9415-00bcd3c36449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:09.188855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe92c1d-5ad1-48fe-9415-00bcd3c36449 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:09.189125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe92c1d-5ad1-48fe-9415-00bcd3c36449 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:09.189385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe92c1d-5ad1-48fe-9415-00bcd3c36449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:09.236826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:09.241821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a96466-5c95-4847-991b-1398b93eef8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:09.243329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a96466-5c95-4847-991b-1398b93eef8e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:09.254971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe92c1d-5ad1-48fe-9415-00bcd3c36449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:09.256460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe92c1d-5ad1-48fe-9415-00bcd3c36449 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:09.272495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:09.342331 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 235/448] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 16:32:09.347914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a96466-5c95-4847-991b-1398b93eef8e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:09.356660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 214/449] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 16:32:09.362308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe92c1d-5ad1-48fe-9415-00bcd3c36449 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:09.440988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:09.448229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a96466-5c95-4847-991b-1398b93eef8e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:09.460190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe92c1d-5ad1-48fe-9415-00bcd3c36449 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:09.476573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:09.482471 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85c503-15a1-45e5-b292-5efb8418f9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:09.483084 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85c503-15a1-45e5-b292-5efb8418f9ae None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:09.483357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85c503-15a1-45e5-b292-5efb8418f9ae None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:09.483620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85c503-15a1-45e5-b292-5efb8418f9ae None admin] PATH_INFO: `/v3/projects/5796807ec0ad4cab845934a9df6b9689` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:09.484728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85c503-15a1-45e5-b292-5efb8418f9ae None admin] RBAC: Authorizing `identity:get_project(project_id=5796807ec0ad4cab845934a9df6b9689)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:09.486237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85c503-15a1-45e5-b292-5efb8418f9ae None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:09.487830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 236/450] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:09 2022] GET /identity/v3/projects/5796807ec0ad4cab845934a9df6b9689 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:09.493221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85c503-15a1-45e5-b292-5efb8418f9ae None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:09.493907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fa355-ddd9-465f-b2cc-984587f3cd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:09.494549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fa355-ddd9-465f-b2cc-984587f3cd77 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:09.494820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fa355-ddd9-465f-b2cc-984587f3cd77 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:09.495082 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fa355-ddd9-465f-b2cc-984587f3cd77 None admin] PATH_INFO: `/v3/projects/e0a53b8ab06c430ea7305247e9df6f92` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:09.496170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fa355-ddd9-465f-b2cc-984587f3cd77 None admin] RBAC: Authorizing `identity:get_project(project_id=e0a53b8ab06c430ea7305247e9df6f92)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:09.497568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fa355-ddd9-465f-b2cc-984587f3cd77 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:09.499184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 215/451] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:32:09 2022] GET /identity/v3/projects/e0a53b8ab06c430ea7305247e9df6f92 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:09.504620 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fa355-ddd9-465f-b2cc-984587f3cd77 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:09.595470 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85c503-15a1-45e5-b292-5efb8418f9ae None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:09.603168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fa355-ddd9-465f-b2cc-984587f3cd77 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:09.627243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d180d5-55b4-4b60-a48b-3697a975b54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:09.627857 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d180d5-55b4-4b60-a48b-3697a975b54d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:09.628136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d180d5-55b4-4b60-a48b-3697a975b54d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:09.628424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d180d5-55b4-4b60-a48b-3697a975b54d None admin] PATH_INFO: `/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:09.629670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d180d5-55b4-4b60-a48b-3697a975b54d None admin] RBAC: Authorizing `identity:get_group(group_id=9908b486ccf84e6ea374363f9ae8c5e5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:09.631622 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d180d5-55b4-4b60-a48b-3697a975b54d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:09.633184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 237/452] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:09 2022] GET /identity/v3/groups/9908b486ccf84e6ea374363f9ae8c5e5 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:09.639056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d180d5-55b4-4b60-a48b-3697a975b54d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:09.642483 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6797bc-4974-41ba-b76f-0a749edd0d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:09.643103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6797bc-4974-41ba-b76f-0a749edd0d5d None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:09.643379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6797bc-4974-41ba-b76f-0a749edd0d5d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:09.643650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6797bc-4974-41ba-b76f-0a749edd0d5d None admin] PATH_INFO: `/v3/groups/ae7fd0887fc946ec8845d5b128694719` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:09.644861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6797bc-4974-41ba-b76f-0a749edd0d5d None admin] RBAC: Authorizing `identity:get_group(group_id=ae7fd0887fc946ec8845d5b128694719)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:09.646312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6797bc-4974-41ba-b76f-0a749edd0d5d None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:09.648422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 216/453] 10.222.0.79 () {56 vars in 1180 bytes} [Tue Aug 9 16:32:09 2022] GET /identity/v3/groups/ae7fd0887fc946ec8845d5b128694719 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:09.654016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6797bc-4974-41ba-b76f-0a749edd0d5d None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:09.671532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:09.703543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:09.737756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d180d5-55b4-4b60-a48b-3697a975b54d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:09.754709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6797bc-4974-41ba-b76f-0a749edd0d5d None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:09.772117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d3f801-68d3-4056-9aec-ab4b65e47721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:09.772716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3f801-68d3-4056-9aec-ab4b65e47721 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:09.772996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3f801-68d3-4056-9aec-ab4b65e47721 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:09.773264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d3f801-68d3-4056-9aec-ab4b65e47721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:09.774006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3f801-68d3-4056-9aec-ab4b65e47721 None admin] RBAC: Adding query filter params (group.id=9908b486ccf84e6ea374363f9ae8c5e5, role.id=20d35b10934447938237c0db5202bbfb, scope.project.id=5796807ec0ad4cab845934a9df6b9689) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:09.774315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3f801-68d3-4056-9aec-ab4b65e47721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:09.775743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d3f801-68d3-4056-9aec-ab4b65e47721 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:09.781589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 238/454] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Aug 9 16:32:09 2022] GET /identity/v3/role_assignments?group.id=9908b486ccf84e6ea374363f9ae8c5e5&scope.project.id=5796807ec0ad4cab845934a9df6b9689&role.id=20d35b10934447938237c0db5202bbfb => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:09.788052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a620da-50ee-4d3a-b9ca-d1f327a91873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:09.788661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a620da-50ee-4d3a-b9ca-d1f327a91873 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:09.788935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a620da-50ee-4d3a-b9ca-d1f327a91873 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:09.789200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a620da-50ee-4d3a-b9ca-d1f327a91873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:09.789932 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a620da-50ee-4d3a-b9ca-d1f327a91873 None admin] RBAC: Adding query filter params (group.id=ae7fd0887fc946ec8845d5b128694719, role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=e0a53b8ab06c430ea7305247e9df6f92) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:09.790261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a620da-50ee-4d3a-b9ca-d1f327a91873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:09.791723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a620da-50ee-4d3a-b9ca-d1f327a91873 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:09.798455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 217/455] 10.222.0.79 () {56 vars in 1376 bytes} [Tue Aug 9 16:32:09 2022] GET /identity/v3/role_assignments?group.id=ae7fd0887fc946ec8845d5b128694719&scope.project.id=e0a53b8ab06c430ea7305247e9df6f92&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:12.326103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5beab03-7135-4c59-8038-f99b4cebb02b None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:12.327168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5beab03-7135-4c59-8038-f99b4cebb02b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:12.327647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5beab03-7135-4c59-8038-f99b4cebb02b None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:12.329491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 239/456] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:12.335359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1635-5624-4d47-9b71-792880039789 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:12.335787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1635-5624-4d47-9b71-792880039789 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:12.335959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d1635-5624-4d47-9b71-792880039789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:12.362974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ab4d1635-5624-4d47-9b71-792880039789 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:12.364179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ab4d1635-5624-4d47-9b71-792880039789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:12.422220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c30f8-d414-49fd-9104-7b9b6ac2d423 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:12.422836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c30f8-d414-49fd-9104-7b9b6ac2d423 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:12.423315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c30f8-d414-49fd-9104-7b9b6ac2d423 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:12.424949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 240/457] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:12.430194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9e12f-17e7-4170-b8c3-292bac1de695 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:12.430821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9e12f-17e7-4170-b8c3-292bac1de695 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:12.431398 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9e12f-17e7-4170-b8c3-292bac1de695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:12.448944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 218/458] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:12.462391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-94d9e12f-17e7-4170-b8c3-292bac1de695 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:12.463845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-94d9e12f-17e7-4170-b8c3-292bac1de695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:12.475754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d1635-5624-4d47-9b71-792880039789 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:12.493654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:12.553021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 241/459] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:12.590017 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9e12f-17e7-4170-b8c3-292bac1de695 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:12.645899 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d1635-5624-4d47-9b71-792880039789 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:12.702323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0dd8f-497c-4cbb-b254-31336564cc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:12.702945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0dd8f-497c-4cbb-b254-31336564cc91 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:12.703220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0dd8f-497c-4cbb-b254-31336564cc91 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:12.703483 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0dd8f-497c-4cbb-b254-31336564cc91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:12.704427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:12.713915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0dd8f-497c-4cbb-b254-31336564cc91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:12.715831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0dd8f-497c-4cbb-b254-31336564cc91 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-1fe0dd8f-497c-4cbb-b254-31336564cc91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:12.731044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:12.732015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:12.732863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:12.732863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:12.732863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:12.732863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 219/460] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:12.738364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0dd8f-497c-4cbb-b254-31336564cc91 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:12.771881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9e12f-17e7-4170-b8c3-292bac1de695 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:12.839404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bd88d-f950-4a6d-bc39-c72584cd8d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:12.840334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bd88d-f950-4a6d-bc39-c72584cd8d57 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:12.840556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bd88d-f950-4a6d-bc39-c72584cd8d57 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:12.840921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bd88d-f950-4a6d-bc39-c72584cd8d57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:12.853594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060bd88d-f950-4a6d-bc39-c72584cd8d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:12.855206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060bd88d-f950-4a6d-bc39-c72584cd8d57 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:12.862310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-060bd88d-f950-4a6d-bc39-c72584cd8d57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:12.890798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:12.893338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:12.894647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:12.894647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:12.894647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:12.894647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 242/461] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:12.900728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bd88d-f950-4a6d-bc39-c72584cd8d57 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:12.930629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0dd8f-497c-4cbb-b254-31336564cc91 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:12.981586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298e18c-70ed-4a41-a0c1-4455ae9b98bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:12.982238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298e18c-70ed-4a41-a0c1-4455ae9b98bc None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:12.982515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298e18c-70ed-4a41-a0c1-4455ae9b98bc None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:12.982780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298e18c-70ed-4a41-a0c1-4455ae9b98bc None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:12.983424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298e18c-70ed-4a41-a0c1-4455ae9b98bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:12.983721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298e18c-70ed-4a41-a0c1-4455ae9b98bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:12.985253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298e18c-70ed-4a41-a0c1-4455ae9b98bc None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:13.010764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 220/462] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:13.015730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298e18c-70ed-4a41-a0c1-4455ae9b98bc None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:13.071994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bd88d-f950-4a6d-bc39-c72584cd8d57 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:13.086899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:13.108310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8741c90-6d55-4f71-a8f2-4ba0d47a147e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:13.108835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8741c90-6d55-4f71-a8f2-4ba0d47a147e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:13.109113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8741c90-6d55-4f71-a8f2-4ba0d47a147e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:13.109386 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8741c90-6d55-4f71-a8f2-4ba0d47a147e None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:13.110057 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8741c90-6d55-4f71-a8f2-4ba0d47a147e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:13.110362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8741c90-6d55-4f71-a8f2-4ba0d47a147e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:13.111857 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8741c90-6d55-4f71-a8f2-4ba0d47a147e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:13.125009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 243/463] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:13.126298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36613b-ce5c-4ef9-9966-771fc889e323 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:13.127060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36613b-ce5c-4ef9-9966-771fc889e323 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:13.127402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36613b-ce5c-4ef9-9966-771fc889e323 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:13.128908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 244/464] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:13.133879 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51175bf5-8fcb-4555-a971-4440c61384da None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:13.134238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51175bf5-8fcb-4555-a971-4440c61384da None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:13.134510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51175bf5-8fcb-4555-a971-4440c61384da None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:13.138128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 245/465] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:13.138128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a085-14f7-4ec7-b425-af2c322626e5 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:13.138128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a085-14f7-4ec7-b425-af2c322626e5 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:13.138128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498a085-14f7-4ec7-b425-af2c322626e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:13.145162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298e18c-70ed-4a41-a0c1-4455ae9b98bc None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:13.155477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-2498a085-14f7-4ec7-b425-af2c322626e5 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:13.157292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2498a085-14f7-4ec7-b425-af2c322626e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:13.168759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:13.180610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e3ffc-4cb0-41a0-bd15-0e30b8fdfdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:13.181207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e3ffc-4cb0-41a0-bd15-0e30b8fdfdcf None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:13.181486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e3ffc-4cb0-41a0-bd15-0e30b8fdfdcf None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:13.181780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e3ffc-4cb0-41a0-bd15-0e30b8fdfdcf None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:13.182425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e3ffc-4cb0-41a0-bd15-0e30b8fdfdcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:13.183876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e3ffc-4cb0-41a0-bd15-0e30b8fdfdcf None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:13.226546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 246/466] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:13.227356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498a085-14f7-4ec7-b425-af2c322626e5 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:13.276519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 221/467] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:32:13 2022] POST /identity/v3/users => generated 246 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:32:13.277945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fa349-281f-4fc3-b107-f0d4f218412d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:13.278499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fa349-281f-4fc3-b107-f0d4f218412d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:13.278845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fa349-281f-4fc3-b107-f0d4f218412d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:13.300681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c07fa349-281f-4fc3-b107-f0d4f218412d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:13.302344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c07fa349-281f-4fc3-b107-f0d4f218412d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:13.353373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498a085-14f7-4ec7-b425-af2c322626e5 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:13.356765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:13.397204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f984ec-06dc-4c19-ae21-a60c4a24c342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:13.397828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f984ec-06dc-4c19-ae21-a60c4a24c342 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:13.398205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f984ec-06dc-4c19-ae21-a60c4a24c342 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:13.398547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f984ec-06dc-4c19-ae21-a60c4a24c342 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:13.399267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f984ec-06dc-4c19-ae21-a60c4a24c342 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:13.399772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 222/468] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:13.401260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fa349-281f-4fc3-b107-f0d4f218412d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:13.407366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f984ec-06dc-4c19-ae21-a60c4a24c342 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:13.501007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 247/469] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:32:13 2022] POST /identity/v3/users => generated 248 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:32:13.503437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f984ec-06dc-4c19-ae21-a60c4a24c342 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:13.592292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fa349-281f-4fc3-b107-f0d4f218412d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:13.620829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:13.655823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f984ec-06dc-4c19-ae21-a60c4a24c342 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:13.701208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3004cf9-0a37-44e7-9404-95c6f19eb61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:13.704077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3004cf9-0a37-44e7-9404-95c6f19eb61d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:13.704531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3004cf9-0a37-44e7-9404-95c6f19eb61d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:13.704937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3004cf9-0a37-44e7-9404-95c6f19eb61d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:13.718762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda096b6-6756-4ea8-8bf3-a0ac4333c98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:13.719714 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda096b6-6756-4ea8-8bf3-a0ac4333c98b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:13.719831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda096b6-6756-4ea8-8bf3-a0ac4333c98b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:13.720055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda096b6-6756-4ea8-8bf3-a0ac4333c98b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:13.722174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3004cf9-0a37-44e7-9404-95c6f19eb61d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:13.724087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3004cf9-0a37-44e7-9404-95c6f19eb61d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-d3004cf9-0a37-44e7-9404-95c6f19eb61d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:13.748715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:13.749661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:13.750536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:13.750536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:13.750536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:13.751161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 248/470] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:13.752396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda096b6-6756-4ea8-8bf3-a0ac4333c98b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:13.760031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda096b6-6756-4ea8-8bf3-a0ac4333c98b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:13.760232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3004cf9-0a37-44e7-9404-95c6f19eb61d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-cda096b6-6756-4ea8-8bf3-a0ac4333c98b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:13.776387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:13.777328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:13.778203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:13.778203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:13.778203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:13.779349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 223/471] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:13.785530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda096b6-6756-4ea8-8bf3-a0ac4333c98b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:13.789527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:13.961905 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3004cf9-0a37-44e7-9404-95c6f19eb61d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:13.963676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda096b6-6756-4ea8-8bf3-a0ac4333c98b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:14.005609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24197d8c-0c7d-4c2f-a129-276cf820db7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:14.007120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24197d8c-0c7d-4c2f-a129-276cf820db7c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:14.007494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9809c-1029-4106-ac96-e905c28dbe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:14.007833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24197d8c-0c7d-4c2f-a129-276cf820db7c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:14.008093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9809c-1029-4106-ac96-e905c28dbe07 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:14.008375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9809c-1029-4106-ac96-e905c28dbe07 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:14.008644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9809c-1029-4106-ac96-e905c28dbe07 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:14.008909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24197d8c-0c7d-4c2f-a129-276cf820db7c None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:14.009289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9809c-1029-4106-ac96-e905c28dbe07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:14.009590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9809c-1029-4106-ac96-e905c28dbe07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:14.010161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24197d8c-0c7d-4c2f-a129-276cf820db7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:14.010666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24197d8c-0c7d-4c2f-a129-276cf820db7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:14.011133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9809c-1029-4106-ac96-e905c28dbe07 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:14.012402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24197d8c-0c7d-4c2f-a129-276cf820db7c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:14.026566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 224/472] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:14.028096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc5fd2-ac7a-4925-a3db-e40d0fec047e None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:14.028601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc5fd2-ac7a-4925-a3db-e40d0fec047e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:14.029066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc5fd2-ac7a-4925-a3db-e40d0fec047e None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:14.030801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 225/473] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:14.032438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 249/474] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:14.033221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc5fd2-ac7a-4925-a3db-e40d0fec047e None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:14.039169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9809c-1029-4106-ac96-e905c28dbe07 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:14.061118 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:14.134339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:14.164869 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9809c-1029-4106-ac96-e905c28dbe07 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:14.182619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc5fd2-ac7a-4925-a3db-e40d0fec047e None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:14.201851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3842aa-6466-4a36-8c32-f962d668ec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:14.202538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3842aa-6466-4a36-8c32-f962d668ec6e None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:14.202823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3842aa-6466-4a36-8c32-f962d668ec6e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:14.203095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3842aa-6466-4a36-8c32-f962d668ec6e None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:14.203789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3842aa-6466-4a36-8c32-f962d668ec6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:14.205419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3842aa-6466-4a36-8c32-f962d668ec6e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:14.224701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247f4ee-4ee8-48d2-8f92-bedd6acd6902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:14.225432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247f4ee-4ee8-48d2-8f92-bedd6acd6902 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:14.225780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247f4ee-4ee8-48d2-8f92-bedd6acd6902 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:14.226164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247f4ee-4ee8-48d2-8f92-bedd6acd6902 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:14.226870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247f4ee-4ee8-48d2-8f92-bedd6acd6902 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:14.228375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247f4ee-4ee8-48d2-8f92-bedd6acd6902 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:14.322178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 250/475] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:32:14 2022] POST /identity/v3/users => generated 249 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:32:14.331810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b021d2-96c0-4e1a-bf8a-d7c740c7f05e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:14.332099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b021d2-96c0-4e1a-bf8a-d7c740c7f05e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:14.332372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b021d2-96c0-4e1a-bf8a-d7c740c7f05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:14.359236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 226/476] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:32:14 2022] POST /identity/v3/users => generated 248 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:32:14.363907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-00b021d2-96c0-4e1a-bf8a-d7c740c7f05e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:14.365179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-00b021d2-96c0-4e1a-bf8a-d7c740c7f05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:14.454458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 251/477] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:14.493414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247f4ee-4ee8-48d2-8f92-bedd6acd6902 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:14.603498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:14.628203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247f4ee-4ee8-48d2-8f92-bedd6acd6902 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:14.664247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bffccd-57b9-4b30-b09e-e5b072f9fa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:14.664848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bffccd-57b9-4b30-b09e-e5b072f9fa15 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:14.665124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bffccd-57b9-4b30-b09e-e5b072f9fa15 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:14.665386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bffccd-57b9-4b30-b09e-e5b072f9fa15 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:14.666081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bffccd-57b9-4b30-b09e-e5b072f9fa15 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:14.667447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bffccd-57b9-4b30-b09e-e5b072f9fa15 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:14.697189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.sql.core [None req-f4bffccd-57b9-4b30-b09e-e5b072f9fa15 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 16:32:14.697189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 16:32:14.697189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [parameters: {'id': '1892727a5d7b4d2a84ed6247d3d6bc50', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 16:32:14.697189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86844) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-f4bffccd-57b9-4b30-b09e-e5b072f9fa15 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application conn.query(q) Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application result.read() Aug 09 16:32:14.714730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return ref.to_dict() Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 16:32:14.716206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application next(self.gen) Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application yield resource Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application next(self.gen) Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise self.value Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application session.commit() Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 16:32:14.717678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self.session.flush() Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application compat.raise_( Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 16:32:14.719705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 16:32:14.720879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application conn.query(q) Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application result.read() Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 16:32:14.726712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [parameters: {'id': '1892727a5d7b4d2a84ed6247d3d6bc50', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:14.727505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 09 16:32:14.728283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:14.729092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 227/478] 10.222.0.79 () {60 vars in 1131 bytes} [Tue Aug 9 16:32:14 2022] POST /identity/v3/roles => generated 144 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:14.729156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b021d2-96c0-4e1a-bf8a-d7c740c7f05e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:14.805558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:14.853176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b021d2-96c0-4e1a-bf8a-d7c740c7f05e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:14.889467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8fc43-c760-476b-b91b-eb8c06ba0fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:14.890397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8fc43-c760-476b-b91b-eb8c06ba0fa2 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:14.890816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8fc43-c760-476b-b91b-eb8c06ba0fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:14.891246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8fc43-c760-476b-b91b-eb8c06ba0fa2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:14.906875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8fc43-c760-476b-b91b-eb8c06ba0fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-84e8fc43-c760-476b-b91b-eb8c06ba0fa2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:14.911707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:14.912796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 09 16:32:14.913769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:14.915979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 252/479] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:14.921229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bffccd-57b9-4b30-b09e-e5b072f9fa15 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:15.091110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bffccd-57b9-4b30-b09e-e5b072f9fa15 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:15.137377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381923fb-bc4f-438f-b5ab-9471df126f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:15.138143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381923fb-bc4f-438f-b5ab-9471df126f67 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:15.138468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381923fb-bc4f-438f-b5ab-9471df126f67 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:15.138819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381923fb-bc4f-438f-b5ab-9471df126f67 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:15.139510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381923fb-bc4f-438f-b5ab-9471df126f67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:15.139807 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381923fb-bc4f-438f-b5ab-9471df126f67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:15.141161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381923fb-bc4f-438f-b5ab-9471df126f67 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:15.154097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 228/480] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:15.842420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f969c-edd0-45d0-b287-207a3f1c87b6 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:15.843407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f969c-edd0-45d0-b287-207a3f1c87b6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:15.843884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f969c-edd0-45d0-b287-207a3f1c87b6 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:15.845747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 253/481] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:15.851523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e4b97-4395-402f-8d0b-c7adc8f93fd2 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:15.852001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e4b97-4395-402f-8d0b-c7adc8f93fd2 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:15.852077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e4b97-4395-402f-8d0b-c7adc8f93fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:15.858274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:15.875422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-a52e4b97-4395-402f-8d0b-c7adc8f93fd2 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:15.876606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a52e4b97-4395-402f-8d0b-c7adc8f93fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:15.954493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 229/482] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:16.001156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f969c-edd0-45d0-b287-207a3f1c87b6 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:16.143437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef63d5-2412-4d84-842b-20d485391ee4 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:16.143785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef63d5-2412-4d84-842b-20d485391ee4 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:16.143903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef63d5-2412-4d84-842b-20d485391ee4 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:16.145471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 230/483] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:16.147625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f969c-edd0-45d0-b287-207a3f1c87b6 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:16.150732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab736b-1849-4b91-96d2-a6d75c926501 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:16.151018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab736b-1849-4b91-96d2-a6d75c926501 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:16.151294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab736b-1849-4b91-96d2-a6d75c926501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:16.170336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-41ab736b-1849-4b91-96d2-a6d75c926501 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:16.171535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-41ab736b-1849-4b91-96d2-a6d75c926501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:16.175764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:16.207395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92d29c-54e3-409f-a867-89bbc0023bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:16.208014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92d29c-54e3-409f-a867-89bbc0023bd5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:16.208287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92d29c-54e3-409f-a867-89bbc0023bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:16.208554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92d29c-54e3-409f-a867-89bbc0023bd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:16.225573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92d29c-54e3-409f-a867-89bbc0023bd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-2a92d29c-54e3-409f-a867-89bbc0023bd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:16.230246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:16.231135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:16.231919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 254/484] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:16.236933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92d29c-54e3-409f-a867-89bbc0023bd5 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:16.293095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 231/485] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:32:16.341348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab736b-1849-4b91-96d2-a6d75c926501 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:16.374041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92d29c-54e3-409f-a867-89bbc0023bd5 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:16.421799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:16.423245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5a2b7-939a-4f2f-8491-75cb4f4f440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:16.424285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5a2b7-939a-4f2f-8491-75cb4f4f440a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:16.424562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5a2b7-939a-4f2f-8491-75cb4f4f440a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:16.424964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5a2b7-939a-4f2f-8491-75cb4f4f440a None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:16.425788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5a2b7-939a-4f2f-8491-75cb4f4f440a None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:16.426116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5a2b7-939a-4f2f-8491-75cb4f4f440a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:16.427497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5a2b7-939a-4f2f-8491-75cb4f4f440a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:16.446028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 255/486] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:16.452096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5a2b7-939a-4f2f-8491-75cb4f4f440a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:16.478555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab736b-1849-4b91-96d2-a6d75c926501 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:16.525373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990be99-c971-4c95-afce-77207c560c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:16.526023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990be99-c971-4c95-afce-77207c560c7b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:16.526304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990be99-c971-4c95-afce-77207c560c7b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:16.529270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c990be99-c971-4c95-afce-77207c560c7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:16.549574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:16.554268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c990be99-c971-4c95-afce-77207c560c7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-c990be99-c971-4c95-afce-77207c560c7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:16.560590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:16.571534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 232/487] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:16.572330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990be99-c971-4c95-afce-77207c560c7b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:16.629565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5a2b7-939a-4f2f-8491-75cb4f4f440a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:16.673512 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaf98e-2292-4e43-9c32-3b60a0751207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:16.675613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaf98e-2292-4e43-9c32-3b60a0751207 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:16.675907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaf98e-2292-4e43-9c32-3b60a0751207 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:16.676181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaf98e-2292-4e43-9c32-3b60a0751207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:16.704030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaf98e-2292-4e43-9c32-3b60a0751207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:16.705492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaf98e-2292-4e43-9c32-3b60a0751207 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-7ebaf98e-2292-4e43-9c32-3b60a0751207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:16.732420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:16.733403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:16.734268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:16.734268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:16.734268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:16.734268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 256/488] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:16.738897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaf98e-2292-4e43-9c32-3b60a0751207 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:16.746647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c990be99-c971-4c95-afce-77207c560c7b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:16.771977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:16.790709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912a5b0-7e5b-453a-b2a6-4daa0242d9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:16.791356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912a5b0-7e5b-453a-b2a6-4daa0242d9ab None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:16.791635 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912a5b0-7e5b-453a-b2a6-4daa0242d9ab None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:16.791900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912a5b0-7e5b-453a-b2a6-4daa0242d9ab None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:16.792797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912a5b0-7e5b-453a-b2a6-4daa0242d9ab None admin] RBAC: Adding query filter params (name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:16.793102 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912a5b0-7e5b-453a-b2a6-4daa0242d9ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:16.796853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912a5b0-7e5b-453a-b2a6-4daa0242d9ab None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:16.812979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 233/489] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:16.823371 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912a5b0-7e5b-453a-b2a6-4daa0242d9ab None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:16.894784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:16.897802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaf98e-2292-4e43-9c32-3b60a0751207 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:16.933903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501fe71-6b76-4603-8729-b656501e7118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:16.934452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501fe71-6b76-4603-8729-b656501e7118 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:16.934736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501fe71-6b76-4603-8729-b656501e7118 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:16.935203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501fe71-6b76-4603-8729-b656501e7118 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:16.935949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501fe71-6b76-4603-8729-b656501e7118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:16.936317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501fe71-6b76-4603-8729-b656501e7118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:16.937752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501fe71-6b76-4603-8729-b656501e7118 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:16.952383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 257/490] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:16.957550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501fe71-6b76-4603-8729-b656501e7118 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:17.048148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912a5b0-7e5b-453a-b2a6-4daa0242d9ab None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:17.090039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05f149-5f30-4a20-b175-2d0ba488cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:17.090968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05f149-5f30-4a20-b175-2d0ba488cd56 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.091188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05f149-5f30-4a20-b175-2d0ba488cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.091455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05f149-5f30-4a20-b175-2d0ba488cd56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.094593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501fe71-6b76-4603-8729-b656501e7118 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:17.103092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05f149-5f30-4a20-b175-2d0ba488cd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:17.104525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05f149-5f30-4a20-b175-2d0ba488cd56 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-2a05f149-5f30-4a20-b175-2d0ba488cd56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:17.119045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:17.119958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:17.120801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:17.120801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:17.120801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:17.120801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 234/491] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:17.121684 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f360d-f47e-49cd-8021-35fb0ca3360d None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.121684 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f360d-f47e-49cd-8021-35fb0ca3360d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.121822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f360d-f47e-49cd-8021-35fb0ca3360d None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.126912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 235/492] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:17.126912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d7bcc-163a-4541-a29d-fb97af37475f None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.126912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d7bcc-163a-4541-a29d-fb97af37475f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.126912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d7bcc-163a-4541-a29d-fb97af37475f None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.127961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 236/493] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:17.129399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d7bcc-163a-4541-a29d-fb97af37475f None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:17.140048 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d066ce8-aa09-460a-b987-a0272503e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:17.140659 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d066ce8-aa09-460a-b987-a0272503e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.140935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d066ce8-aa09-460a-b987-a0272503e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.141270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d066ce8-aa09-460a-b987-a0272503e7f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.148352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d066ce8-aa09-460a-b987-a0272503e7f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:17.149685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d066ce8-aa09-460a-b987-a0272503e7f8 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-9d066ce8-aa09-460a-b987-a0272503e7f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:17.162132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:17.163552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:17.164449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 258/494] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Aug 9 16:32:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:17.165666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd96f72-d3c3-47fd-8273-4c9af16e2b90 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.165666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd96f72-d3c3-47fd-8273-4c9af16e2b90 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.165666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd96f72-d3c3-47fd-8273-4c9af16e2b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.174940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:17.192766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-cbd96f72-d3c3-47fd-8273-4c9af16e2b90 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:17.192766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cbd96f72-d3c3-47fd-8273-4c9af16e2b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:17.192766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:17.249254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d7bcc-163a-4541-a29d-fb97af37475f None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:17.252875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 259/495] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:17.254123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad5ec7-7a3c-4def-86c7-6854c244a7c1 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.254401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad5ec7-7a3c-4def-86c7-6854c244a7c1 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.254664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad5ec7-7a3c-4def-86c7-6854c244a7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.272933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-46ad5ec7-7a3c-4def-86c7-6854c244a7c1 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:17.274124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-46ad5ec7-7a3c-4def-86c7-6854c244a7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:17.295547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f1b42-32e7-4dc7-b5e6-a0e01c4fa5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:17.296487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f1b42-32e7-4dc7-b5e6-a0e01c4fa5e9 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.296957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f1b42-32e7-4dc7-b5e6-a0e01c4fa5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.297527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f1b42-32e7-4dc7-b5e6-a0e01c4fa5e9 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.298494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f1b42-32e7-4dc7-b5e6-a0e01c4fa5e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:17.299002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f1b42-32e7-4dc7-b5e6-a0e01c4fa5e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:17.300651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f1b42-32e7-4dc7-b5e6-a0e01c4fa5e9 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:17.313513 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 237/496] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:17.314614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f1b42-32e7-4dc7-b5e6-a0e01c4fa5e9 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:17.345011 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 260/497] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:17.346093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad5ec7-7a3c-4def-86c7-6854c244a7c1 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:17.418090 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:17.427954 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:17.433459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f1b42-32e7-4dc7-b5e6-a0e01c4fa5e9 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:17.449279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad5ec7-7a3c-4def-86c7-6854c244a7c1 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:17.469003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aff625-68c2-446a-9bbb-0ff1090a4982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:17.469604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aff625-68c2-446a-9bbb-0ff1090a4982 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.469876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aff625-68c2-446a-9bbb-0ff1090a4982 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.470166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aff625-68c2-446a-9bbb-0ff1090a4982 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.471477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aff625-68c2-446a-9bbb-0ff1090a4982 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:17.471477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aff625-68c2-446a-9bbb-0ff1090a4982 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:17.472726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aff625-68c2-446a-9bbb-0ff1090a4982 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:17.485066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf06f48-e857-4f0c-84a2-6992a618afc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:17.485701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf06f48-e857-4f0c-84a2-6992a618afc7 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.485998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf06f48-e857-4f0c-84a2-6992a618afc7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.486272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf06f48-e857-4f0c-84a2-6992a618afc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.496839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf06f48-e857-4f0c-84a2-6992a618afc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-1cf06f48-e857-4f0c-84a2-6992a618afc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:17.501520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:17.502456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 261/498] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:17.508438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf06f48-e857-4f0c-84a2-6992a618afc7 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:17.516565 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 238/499] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Aug 9 16:32:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:17.517400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aff625-68c2-446a-9bbb-0ff1090a4982 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:17.616360 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf06f48-e857-4f0c-84a2-6992a618afc7 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:17.631106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aff625-68c2-446a-9bbb-0ff1090a4982 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:17.645499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:17.651783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e41e9b-e237-4340-aaad-c21802f82269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:17.652397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e41e9b-e237-4340-aaad-c21802f82269 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.652672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e41e9b-e237-4340-aaad-c21802f82269 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.652938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e41e9b-e237-4340-aaad-c21802f82269 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.660593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e41e9b-e237-4340-aaad-c21802f82269 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:17.662236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e41e9b-e237-4340-aaad-c21802f82269 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:17.667486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a0ded-6691-42d6-8628-f3e1478a9b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:17.668154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a0ded-6691-42d6-8628-f3e1478a9b8e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.668427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a0ded-6691-42d6-8628-f3e1478a9b8e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.668690 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a0ded-6691-42d6-8628-f3e1478a9b8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-d4e41e9b-e237-4340-aaad-c21802f82269 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:17.673468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:17.674397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:17.675280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 262/500] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:17.675820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e41e9b-e237-4340-aaad-c21802f82269 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:17.684896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a0ded-6691-42d6-8628-f3e1478a9b8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-047a0ded-6691-42d6-8628-f3e1478a9b8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:17.689654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:17.690614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 239/501] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:17.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a0ded-6691-42d6-8628-f3e1478a9b8e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:17.711732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:17.789981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e41e9b-e237-4340-aaad-c21802f82269 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:17.799352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a0ded-6691-42d6-8628-f3e1478a9b8e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:17.824704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f6a07-96ce-41e0-82d9-c35300df489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:17.825301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f6a07-96ce-41e0-82d9-c35300df489b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.825570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f6a07-96ce-41e0-82d9-c35300df489b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.825834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f6a07-96ce-41e0-82d9-c35300df489b None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.826515 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f6a07-96ce-41e0-82d9-c35300df489b None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:17.826810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f6a07-96ce-41e0-82d9-c35300df489b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:17.828186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f6a07-96ce-41e0-82d9-c35300df489b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:17.841981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 263/502] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:17.841981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f6a07-96ce-41e0-82d9-c35300df489b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:17.844082 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d8151-58e8-431b-a1ed-5c950e58a575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:17.844691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d8151-58e8-431b-a1ed-5c950e58a575 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.844965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d8151-58e8-431b-a1ed-5c950e58a575 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.845235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d8151-58e8-431b-a1ed-5c950e58a575 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.892354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:17.918828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d8151-58e8-431b-a1ed-5c950e58a575 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:17.920346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d8151-58e8-431b-a1ed-5c950e58a575 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:17.934940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:17.938626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f6a07-96ce-41e0-82d9-c35300df489b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:17.972498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e271fe-98aa-4f12-804a-660efce0cc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:17.973047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e271fe-98aa-4f12-804a-660efce0cc07 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:17.973322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e271fe-98aa-4f12-804a-660efce0cc07 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:17.973586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e271fe-98aa-4f12-804a-660efce0cc07 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:17.974320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e271fe-98aa-4f12-804a-660efce0cc07 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:17.974622 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e271fe-98aa-4f12-804a-660efce0cc07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:17.976069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e271fe-98aa-4f12-804a-660efce0cc07 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:18.012238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 264/503] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Aug 9 16:32:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:18.013053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e271fe-98aa-4f12-804a-660efce0cc07 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.019661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 240/504] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:18.020489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d8151-58e8-431b-a1ed-5c950e58a575 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.107644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e271fe-98aa-4f12-804a-660efce0cc07 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.118250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:18.120213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d8151-58e8-431b-a1ed-5c950e58a575 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.134701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:18.141410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d7171-8c36-4018-a073-08a0b7bd8c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:18.142025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7171-8c36-4018-a073-08a0b7bd8c05 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.142300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7171-8c36-4018-a073-08a0b7bd8c05 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.142565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7171-8c36-4018-a073-08a0b7bd8c05 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.143212 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d7171-8c36-4018-a073-08a0b7bd8c05 None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:18.143507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d7171-8c36-4018-a073-08a0b7bd8c05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:18.144961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d7171-8c36-4018-a073-08a0b7bd8c05 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:18.152412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 265/505] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:18.153527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d642546-40ef-41be-83f7-f6b13033c8eb None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.153759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f676178-2c65-4a4f-aa6f-b43d24473282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:18.153876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d642546-40ef-41be-83f7-f6b13033c8eb None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.154085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d642546-40ef-41be-83f7-f6b13033c8eb None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.154396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f676178-2c65-4a4f-aa6f-b43d24473282 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.154674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f676178-2c65-4a4f-aa6f-b43d24473282 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.155057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f676178-2c65-4a4f-aa6f-b43d24473282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.155544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 266/506] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:18.156137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d642546-40ef-41be-83f7-f6b13033c8eb None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.167942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f676178-2c65-4a4f-aa6f-b43d24473282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:18.169446 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f676178-2c65-4a4f-aa6f-b43d24473282 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-1f676178-2c65-4a4f-aa6f-b43d24473282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:18.184021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:18.184895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:18.190692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:18.190692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:18.190692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:18.190692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 241/507] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:18.190692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f676178-2c65-4a4f-aa6f-b43d24473282 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.258285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d642546-40ef-41be-83f7-f6b13033c8eb None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.290290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f676178-2c65-4a4f-aa6f-b43d24473282 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.290529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9336e5-749a-4e6f-8b12-e37757894b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:18.291122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9336e5-749a-4e6f-8b12-e37757894b7f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.291397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9336e5-749a-4e6f-8b12-e37757894b7f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.291664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9336e5-749a-4e6f-8b12-e37757894b7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.323117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0659c3-b8b6-40e8-b6bb-66b971a7ef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:18.323721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0659c3-b8b6-40e8-b6bb-66b971a7ef3e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.323997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0659c3-b8b6-40e8-b6bb-66b971a7ef3e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.324263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0659c3-b8b6-40e8-b6bb-66b971a7ef3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.332321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:18.334546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0659c3-b8b6-40e8-b6bb-66b971a7ef3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:18.335979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0659c3-b8b6-40e8-b6bb-66b971a7ef3e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-1a0659c3-b8b6-40e8-b6bb-66b971a7ef3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:18.350733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:18.352088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:18.352930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:18.352930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:18.352930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:18.352930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 242/508] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:18.352930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0659c3-b8b6-40e8-b6bb-66b971a7ef3e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.365209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9336e5-749a-4e6f-8b12-e37757894b7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:18.367293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9336e5-749a-4e6f-8b12-e37757894b7f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:18.372027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:18.452679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0659c3-b8b6-40e8-b6bb-66b971a7ef3e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.466160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 267/509] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:18.467395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4749ab-1ad6-439f-960e-ca0c1548f5dc None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.467696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4749ab-1ad6-439f-960e-ca0c1548f5dc None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.467965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4749ab-1ad6-439f-960e-ca0c1548f5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.486930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-de4749ab-1ad6-439f-960e-ca0c1548f5dc None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:18.488136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-de4749ab-1ad6-439f-960e-ca0c1548f5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:18.492236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1c147-233c-441e-83fe-f94d275ddb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:18.492848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1c147-233c-441e-83fe-f94d275ddb6f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.493120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1c147-233c-441e-83fe-f94d275ddb6f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.493383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1c147-233c-441e-83fe-f94d275ddb6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.503820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1c147-233c-441e-83fe-f94d275ddb6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:18.505385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1c147-233c-441e-83fe-f94d275ddb6f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-a8c1c147-233c-441e-83fe-f94d275ddb6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:18.520739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:18.521679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:18.522981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:18.522981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:18.522981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:18.522981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 243/510] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:18.522981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1c147-233c-441e-83fe-f94d275ddb6f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.561504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 268/511] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:18.562700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4749ab-1ad6-439f-960e-ca0c1548f5dc None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.564956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:18.599165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:18.632253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1c147-233c-441e-83fe-f94d275ddb6f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.659222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4749ab-1ad6-439f-960e-ca0c1548f5dc None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.666836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd563c46-3b50-4eb4-999b-ddef5724cb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:18.667429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd563c46-3b50-4eb4-999b-ddef5724cb37 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.667704 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd563c46-3b50-4eb4-999b-ddef5724cb37 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.668255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd563c46-3b50-4eb4-999b-ddef5724cb37 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.668909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd563c46-3b50-4eb4-999b-ddef5724cb37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:18.669208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd563c46-3b50-4eb4-999b-ddef5724cb37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:18.670633 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd563c46-3b50-4eb4-999b-ddef5724cb37 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:18.682801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 244/512] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:18.683634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd563c46-3b50-4eb4-999b-ddef5724cb37 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.694049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8589f-de04-4d10-ace9-fe982da8404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:18.694644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8589f-de04-4d10-ace9-fe982da8404f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.694917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8589f-de04-4d10-ace9-fe982da8404f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.695181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b8589f-de04-4d10-ace9-fe982da8404f None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.695815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8589f-de04-4d10-ace9-fe982da8404f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:18.696109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8589f-de04-4d10-ace9-fe982da8404f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:18.697473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b8589f-de04-4d10-ace9-fe982da8404f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:18.708346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 269/513] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:18.709133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8589f-de04-4d10-ace9-fe982da8404f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.769548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:18.783363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd563c46-3b50-4eb4-999b-ddef5724cb37 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.807864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b8589f-de04-4d10-ace9-fe982da8404f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.811168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:18.816809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93072d08-cea2-4d66-9890-b9fc2f703d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:18.817424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93072d08-cea2-4d66-9890-b9fc2f703d95 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.820704 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93072d08-cea2-4d66-9890-b9fc2f703d95 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.820704 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93072d08-cea2-4d66-9890-b9fc2f703d95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.828253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93072d08-cea2-4d66-9890-b9fc2f703d95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:18.829834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93072d08-cea2-4d66-9890-b9fc2f703d95 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:18.841455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25049548-e018-4d8e-a2a8-61ce33e5a407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:18.842077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049548-e018-4d8e-a2a8-61ce33e5a407 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.842354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049548-e018-4d8e-a2a8-61ce33e5a407 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.842621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25049548-e018-4d8e-a2a8-61ce33e5a407 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.843250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25049548-e018-4d8e-a2a8-61ce33e5a407 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:18.843544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25049548-e018-4d8e-a2a8-61ce33e5a407 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-93072d08-cea2-4d66-9890-b9fc2f703d95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:18.844702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:18.845572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:18.846462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:18.846462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:18.846462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:18.846659 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25049548-e018-4d8e-a2a8-61ce33e5a407 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:18.851205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 245/514] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:18.852005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93072d08-cea2-4d66-9890-b9fc2f703d95 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.861552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 270/515] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:18.862395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25049548-e018-4d8e-a2a8-61ce33e5a407 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:18.949920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93072d08-cea2-4d66-9890-b9fc2f703d95 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.963614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25049548-e018-4d8e-a2a8-61ce33e5a407 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:18.985230 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08746b5c-b92c-4eaa-90e4-ddfa24c77204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:18.985845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08746b5c-b92c-4eaa-90e4-ddfa24c77204 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:18.986013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:18.986173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08746b5c-b92c-4eaa-90e4-ddfa24c77204 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:18.986448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08746b5c-b92c-4eaa-90e4-ddfa24c77204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:18.996564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08746b5c-b92c-4eaa-90e4-ddfa24c77204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.003736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ac4fd-bba1-4980-9dc9-843bbfb2d4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.003855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08746b5c-b92c-4eaa-90e4-ddfa24c77204 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.003965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ac4fd-bba1-4980-9dc9-843bbfb2d4c9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.003965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ac4fd-bba1-4980-9dc9-843bbfb2d4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.003965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ac4fd-bba1-4980-9dc9-843bbfb2d4c9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.005296 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ac4fd-bba1-4980-9dc9-843bbfb2d4c9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.006779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ac4fd-bba1-4980-9dc9-843bbfb2d4c9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-08746b5c-b92c-4eaa-90e4-ddfa24c77204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:19.011770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:19.012629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:19.013469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:19.013469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:19.013469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:19.013469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 246/516] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:19.013678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08746b5c-b92c-4eaa-90e4-ddfa24c77204 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-b29ac4fd-bba1-4980-9dc9-843bbfb2d4c9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:19.023488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:19.024355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 271/517] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:19.025900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ac4fd-bba1-4980-9dc9-843bbfb2d4c9 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.052156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:19.118911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08746b5c-b92c-4eaa-90e4-ddfa24c77204 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.125103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ac4fd-bba1-4980-9dc9-843bbfb2d4c9 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.153350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cb507-05e2-4a92-aa77-523962874cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.153976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cb507-05e2-4a92-aa77-523962874cfd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.154275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cb507-05e2-4a92-aa77-523962874cfd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.154544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cb507-05e2-4a92-aa77-523962874cfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.158495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aeab73-672b-4232-b719-d8e4e4097649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.159086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aeab73-672b-4232-b719-d8e4e4097649 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.159361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aeab73-672b-4232-b719-d8e4e4097649 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.159625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aeab73-672b-4232-b719-d8e4e4097649 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.160252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aeab73-672b-4232-b719-d8e4e4097649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:19.160547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aeab73-672b-4232-b719-d8e4e4097649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.161008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cb507-05e2-4a92-aa77-523962874cfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.161911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aeab73-672b-4232-b719-d8e4e4097649 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.162462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cb507-05e2-4a92-aa77-523962874cfd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.174220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 272/518] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-031cb507-05e2-4a92-aa77-523962874cfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:19.174357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:19.175134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:19.175974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:19.175974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:19.175974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:19.175974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:19.175974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:19.175974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:19.175974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:19.175974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:19.176383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aeab73-672b-4232-b719-d8e4e4097649 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.176462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 247/519] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:19.176462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cb507-05e2-4a92-aa77-523962874cfd None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.221482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:19.274782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:19.275576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aeab73-672b-4232-b719-d8e4e4097649 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.281194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cb507-05e2-4a92-aa77-523962874cfd None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.308578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cb1d2-c124-4e8f-9606-6a4cc4069397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.309117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cb1d2-c124-4e8f-9606-6a4cc4069397 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.309392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cb1d2-c124-4e8f-9606-6a4cc4069397 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.309658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cb1d2-c124-4e8f-9606-6a4cc4069397 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.316389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730143e-5cd5-49a9-b6b2-fc1400dfb4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.316949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730143e-5cd5-49a9-b6b2-fc1400dfb4a6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.317223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730143e-5cd5-49a9-b6b2-fc1400dfb4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.317489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730143e-5cd5-49a9-b6b2-fc1400dfb4a6 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.318151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730143e-5cd5-49a9-b6b2-fc1400dfb4a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:19.318447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730143e-5cd5-49a9-b6b2-fc1400dfb4a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.319653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cb1d2-c124-4e8f-9606-6a4cc4069397 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.320358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730143e-5cd5-49a9-b6b2-fc1400dfb4a6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.321106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10cb1d2-c124-4e8f-9606-6a4cc4069397 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.331549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 248/520] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:19.332335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730143e-5cd5-49a9-b6b2-fc1400dfb4a6 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-c10cb1d2-c124-4e8f-9606-6a4cc4069397 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:19.332947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:19.333806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 273/521] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:19.335974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cb1d2-c124-4e8f-9606-6a4cc4069397 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.432785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10cb1d2-c124-4e8f-9606-6a4cc4069397 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.433203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730143e-5cd5-49a9-b6b2-fc1400dfb4a6 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.434307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:19.466648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75dff6-0e1d-4855-91b6-c425aa7af4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.467255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75dff6-0e1d-4855-91b6-c425aa7af4b7 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.467530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75dff6-0e1d-4855-91b6-c425aa7af4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.467622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57562cb-864f-4711-a866-11ce546933ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.467800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75dff6-0e1d-4855-91b6-c425aa7af4b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.468194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57562cb-864f-4711-a866-11ce546933ec None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.468472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57562cb-864f-4711-a866-11ce546933ec None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.468742 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57562cb-864f-4711-a866-11ce546933ec None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.469496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57562cb-864f-4711-a866-11ce546933ec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:19.469806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57562cb-864f-4711-a866-11ce546933ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.471239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57562cb-864f-4711-a866-11ce546933ec None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.474963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75dff6-0e1d-4855-91b6-c425aa7af4b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.476424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75dff6-0e1d-4855-91b6-c425aa7af4b7 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-1d75dff6-0e1d-4855-91b6-c425aa7af4b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:19.487895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:19.488833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:19.494859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:19.494859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:19.494859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:19.494859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:19.494859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:19.494859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:19.494859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:19.494859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:19.495373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 249/522] 10.222.0.79 () {56 vars in 1139 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:19.495373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57562cb-864f-4711-a866-11ce546933ec None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.495521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 274/523] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:19.495521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75dff6-0e1d-4855-91b6-c425aa7af4b7 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.508997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:19.589846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75dff6-0e1d-4855-91b6-c425aa7af4b7 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.596949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57562cb-864f-4711-a866-11ce546933ec None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.624172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd98730-aea8-468a-83ef-7b802f5e0003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.624782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd98730-aea8-468a-83ef-7b802f5e0003 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.625061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd98730-aea8-468a-83ef-7b802f5e0003 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.625333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd98730-aea8-468a-83ef-7b802f5e0003 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.626095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd98730-aea8-468a-83ef-7b802f5e0003 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.627634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd98730-aea8-468a-83ef-7b802f5e0003 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.634731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf11884-4199-4f86-8547-14d10deb6e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.635391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf11884-4199-4f86-8547-14d10deb6e05 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.635634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf11884-4199-4f86-8547-14d10deb6e05 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.635902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf11884-4199-4f86-8547-14d10deb6e05 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.636589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf11884-4199-4f86-8547-14d10deb6e05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:19.636888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf11884-4199-4f86-8547-14d10deb6e05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.638355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf11884-4199-4f86-8547-14d10deb6e05 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.657126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 250/524] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:19.657999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf11884-4199-4f86-8547-14d10deb6e05 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.726101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:19.748702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 275/525] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:32:19 2022] POST /identity/v3/users => generated 247 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:32:19.749694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd98730-aea8-468a-83ef-7b802f5e0003 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.754156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf11884-4199-4f86-8547-14d10deb6e05 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.773118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:19.795998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1754dc5-a818-499c-a6fa-0f13fde0fcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.796726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754dc5-a818-499c-a6fa-0f13fde0fcc8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.797076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754dc5-a818-499c-a6fa-0f13fde0fcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.797356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1754dc5-a818-499c-a6fa-0f13fde0fcc8 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.798144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1754dc5-a818-499c-a6fa-0f13fde0fcc8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:19.798450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1754dc5-a818-499c-a6fa-0f13fde0fcc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.799926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1754dc5-a818-499c-a6fa-0f13fde0fcc8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.820649 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 251/526] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:19.825413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1754dc5-a818-499c-a6fa-0f13fde0fcc8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:19.871970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd98730-aea8-468a-83ef-7b802f5e0003 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.909174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0259fa-3be1-414f-8ba5-fd6e13e6728d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.909771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0259fa-3be1-414f-8ba5-fd6e13e6728d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.910099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0259fa-3be1-414f-8ba5-fd6e13e6728d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.910384 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0259fa-3be1-414f-8ba5-fd6e13e6728d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.935079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1754dc5-a818-499c-a6fa-0f13fde0fcc8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:19.972540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:19.979360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d9240-82e4-4d09-a89d-f069c1eef2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:19.979957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d9240-82e4-4d09-a89d-f069c1eef2dc None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:19.980228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d9240-82e4-4d09-a89d-f069c1eef2dc None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:19.980493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d9240-82e4-4d09-a89d-f069c1eef2dc None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:19.981251 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d9240-82e4-4d09-a89d-f069c1eef2dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:19.981478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d9240-82e4-4d09-a89d-f069c1eef2dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.983546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d9240-82e4-4d09-a89d-f069c1eef2dc None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.992276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0259fa-3be1-414f-8ba5-fd6e13e6728d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:19.993739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0259fa-3be1-414f-8ba5-fd6e13e6728d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:19.999575 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 252/527] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:20.000328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d9240-82e4-4d09-a89d-f069c1eef2dc None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:20.020990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:20.111262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d9240-82e4-4d09-a89d-f069c1eef2dc None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:20.112243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 276/528] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:20.113402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0259fa-3be1-414f-8ba5-fd6e13e6728d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:20.151882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3383c79-cd79-4a8b-957b-0db710bbc5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:20.152419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3383c79-cd79-4a8b-957b-0db710bbc5ef None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:20.152700 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3383c79-cd79-4a8b-957b-0db710bbc5ef None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:20.152967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3383c79-cd79-4a8b-957b-0db710bbc5ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:20.153787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3383c79-cd79-4a8b-957b-0db710bbc5ef None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=5d17aa4b23e04290b6c8b1924b0dcf32) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:20.154404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3383c79-cd79-4a8b-957b-0db710bbc5ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:20.155701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3383c79-cd79-4a8b-957b-0db710bbc5ef None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:20.161731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 253/529] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:19 2022] GET /identity/v3/role_assignments?user.id=5d17aa4b23e04290b6c8b1924b0dcf32&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:20.162463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3383c79-cd79-4a8b-957b-0db710bbc5ef None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:20.228758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:20.256668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0259fa-3be1-414f-8ba5-fd6e13e6728d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:20.279295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3383c79-cd79-4a8b-957b-0db710bbc5ef None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:20.284845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:20.304993 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ac137-9b71-4c00-b4cc-3117954855ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:20.305980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ac137-9b71-4c00-b4cc-3117954855ba None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:20.305980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ac137-9b71-4c00-b4cc-3117954855ba None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:20.306175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ac137-9b71-4c00-b4cc-3117954855ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:20.325031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76822a-4112-409e-a041-65f8be0df564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:20.326188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76822a-4112-409e-a041-65f8be0df564 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:20.326188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76822a-4112-409e-a041-65f8be0df564 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:20.326527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76822a-4112-409e-a041-65f8be0df564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:20.327381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76822a-4112-409e-a041-65f8be0df564 None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=33003a9bf4454f5bac2e6ad73544efc4) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:20.327768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76822a-4112-409e-a041-65f8be0df564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:20.330255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f76822a-4112-409e-a041-65f8be0df564 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:20.336556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 254/530] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:20 2022] GET /identity/v3/role_assignments?user.id=33003a9bf4454f5bac2e6ad73544efc4&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:20.337449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76822a-4112-409e-a041-65f8be0df564 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:20.396880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ac137-9b71-4c00-b4cc-3117954855ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:20.399064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ac137-9b71-4c00-b4cc-3117954855ba None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:20.559263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76822a-4112-409e-a041-65f8be0df564 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:20.566240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 277/531] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:20.572435 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ac137-9b71-4c00-b4cc-3117954855ba None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:20.579721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:20.598529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e94d6-4c1d-443d-be53-fc5f48b82042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:20.599307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e94d6-4c1d-443d-be53-fc5f48b82042 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:20.599721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e94d6-4c1d-443d-be53-fc5f48b82042 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:20.599996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e94d6-4c1d-443d-be53-fc5f48b82042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:20.612654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:20.613697 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e94d6-4c1d-443d-be53-fc5f48b82042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:20.615369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e94d6-4c1d-443d-be53-fc5f48b82042 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-743e94d6-4c1d-443d-be53-fc5f48b82042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:20.633679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:20.634828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:20.635870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:20.635870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:20.635870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:20.638243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 255/532] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:20.647854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e94d6-4c1d-443d-be53-fc5f48b82042 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:20.732817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ac137-9b71-4c00-b4cc-3117954855ba None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:20.767104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bddea-79d0-438c-8147-263aaa69420e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:20.767815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bddea-79d0-438c-8147-263aaa69420e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:20.768085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bddea-79d0-438c-8147-263aaa69420e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:20.768353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bddea-79d0-438c-8147-263aaa69420e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:20.786140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bddea-79d0-438c-8147-263aaa69420e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:20.787508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bddea-79d0-438c-8147-263aaa69420e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:20.804405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e94d6-4c1d-443d-be53-fc5f48b82042 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-5e2bddea-79d0-438c-8147-263aaa69420e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:20.820108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:20.821021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:20.821870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:20.821870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:20.821870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:20.827950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 278/533] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:20.833411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bddea-79d0-438c-8147-263aaa69420e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:20.869227 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36461503-e98b-4437-86fd-440d07b3ec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:20.869844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36461503-e98b-4437-86fd-440d07b3ec73 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:20.870145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36461503-e98b-4437-86fd-440d07b3ec73 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:20.870415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36461503-e98b-4437-86fd-440d07b3ec73 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:20.871062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36461503-e98b-4437-86fd-440d07b3ec73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:20.871363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36461503-e98b-4437-86fd-440d07b3ec73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:20.872770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36461503-e98b-4437-86fd-440d07b3ec73 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:20.899352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 256/534] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:20.905259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36461503-e98b-4437-86fd-440d07b3ec73 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:20.938804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:20.976193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:21.018410 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36461503-e98b-4437-86fd-440d07b3ec73 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:21.044242 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bddea-79d0-438c-8147-263aaa69420e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:21.058128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63accf-5270-4dd6-b443-24cc29aed93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:21.059128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63accf-5270-4dd6-b443-24cc29aed93f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:21.059630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63accf-5270-4dd6-b443-24cc29aed93f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:21.060198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63accf-5270-4dd6-b443-24cc29aed93f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:21.067685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63accf-5270-4dd6-b443-24cc29aed93f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:21.070224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63accf-5270-4dd6-b443-24cc29aed93f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-0e63accf-5270-4dd6-b443-24cc29aed93f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:21.086375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:21.087257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:21.088079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:21.088079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:21.088079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:21.088079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:21.088079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:21.088079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:21.088079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:21.088079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:21.090444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 257/535] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:21.095931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63accf-5270-4dd6-b443-24cc29aed93f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:21.134789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce7013-7252-42af-99c3-313ed149f27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:21.135161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce7013-7252-42af-99c3-313ed149f27f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:21.135415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce7013-7252-42af-99c3-313ed149f27f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:21.135684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce7013-7252-42af-99c3-313ed149f27f None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:21.136323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce7013-7252-42af-99c3-313ed149f27f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:21.136712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce7013-7252-42af-99c3-313ed149f27f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:21.138020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce7013-7252-42af-99c3-313ed149f27f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:21.164409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 279/536] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:21.171175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce7013-7252-42af-99c3-313ed149f27f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:21.207363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:21.218585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63accf-5270-4dd6-b443-24cc29aed93f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:21.257537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4a887-d78c-4c82-9e2e-af3468198229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:21.258550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a887-d78c-4c82-9e2e-af3468198229 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:21.259075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a887-d78c-4c82-9e2e-af3468198229 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:21.259536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4a887-d78c-4c82-9e2e-af3468198229 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:21.260446 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4a887-d78c-4c82-9e2e-af3468198229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:21.260960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4a887-d78c-4c82-9e2e-af3468198229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:21.262625 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc4a887-d78c-4c82-9e2e-af3468198229 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:21.275534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 258/537] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:21.281457 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4a887-d78c-4c82-9e2e-af3468198229 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:21.356060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce7013-7252-42af-99c3-313ed149f27f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:21.388725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4a887-d78c-4c82-9e2e-af3468198229 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:21.411201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:21.417280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df2115-b61c-4bec-9ac3-4393ceabe7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:21.417934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2115-b61c-4bec-9ac3-4393ceabe7cd None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:21.424101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2115-b61c-4bec-9ac3-4393ceabe7cd None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:21.424396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2115-b61c-4bec-9ac3-4393ceabe7cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:21.427524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dafe3e-d3de-4ba9-a785-4270a1779e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:21.428366 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dafe3e-d3de-4ba9-a785-4270a1779e86 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:21.429366 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dafe3e-d3de-4ba9-a785-4270a1779e86 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:21.429852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dafe3e-d3de-4ba9-a785-4270a1779e86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:21.430844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dafe3e-d3de-4ba9-a785-4270a1779e86 None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=c2f11dcf2c3c41a0b435a90af74c2e5a) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:21.431391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dafe3e-d3de-4ba9-a785-4270a1779e86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:21.432384 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df2115-b61c-4bec-9ac3-4393ceabe7cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:21.433218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dafe3e-d3de-4ba9-a785-4270a1779e86 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:21.433818 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df2115-b61c-4bec-9ac3-4393ceabe7cd None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:21.439826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 259/538] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:21 2022] GET /identity/v3/role_assignments?user.id=c2f11dcf2c3c41a0b435a90af74c2e5a&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-f5df2115-b61c-4bec-9ac3-4393ceabe7cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:21.458067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:21.459197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:21.460234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:21.460234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:21.460234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:21.460234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:21.460234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:21.460234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:21.460234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:21.460234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:21.462397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 280/539] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:21.467891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dafe3e-d3de-4ba9-a785-4270a1779e86 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:21.504488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:21.638269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dafe3e-d3de-4ba9-a785-4270a1779e86 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:21.678260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6d1d1-0eab-4936-9fb1-4b5f41de73bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:21.679474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6d1d1-0eab-4936-9fb1-4b5f41de73bd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:21.679753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6d1d1-0eab-4936-9fb1-4b5f41de73bd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:21.680026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6d1d1-0eab-4936-9fb1-4b5f41de73bd None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:21.680699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6d1d1-0eab-4936-9fb1-4b5f41de73bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:21.680995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6d1d1-0eab-4936-9fb1-4b5f41de73bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:21.682459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6d1d1-0eab-4936-9fb1-4b5f41de73bd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:21.694422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 260/540] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:21.700921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df2115-b61c-4bec-9ac3-4393ceabe7cd None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:21.820549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df2115-b61c-4bec-9ac3-4393ceabe7cd None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:21.855748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a515a7-b781-4eda-99ef-9b83ffcfcddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:21.856338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a515a7-b781-4eda-99ef-9b83ffcfcddd None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:21.856613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a515a7-b781-4eda-99ef-9b83ffcfcddd None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:21.856878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a515a7-b781-4eda-99ef-9b83ffcfcddd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:21.857592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a515a7-b781-4eda-99ef-9b83ffcfcddd None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=289e7caceab944e9b90f2e9298fb2cf1) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:21.857896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a515a7-b781-4eda-99ef-9b83ffcfcddd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:21.859397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a515a7-b781-4eda-99ef-9b83ffcfcddd None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:21.865207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 281/541] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:21 2022] GET /identity/v3/role_assignments?user.id=289e7caceab944e9b90f2e9298fb2cf1&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:22.165105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70132771-7ec5-4285-b528-ccbfb3dbc45e None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:22.165491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70132771-7ec5-4285-b528-ccbfb3dbc45e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:22.165664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70132771-7ec5-4285-b528-ccbfb3dbc45e None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:22.167300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 261/542] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:22.172728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae21267-61ff-4679-9dfd-669d7e70ed32 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:22.173192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae21267-61ff-4679-9dfd-669d7e70ed32 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:22.173294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae21267-61ff-4679-9dfd-669d7e70ed32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:22.191701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-6ae21267-61ff-4679-9dfd-669d7e70ed32 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:22.192903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6ae21267-61ff-4679-9dfd-669d7e70ed32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:22.246083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:22.270066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 282/543] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:22.288066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70132771-7ec5-4285-b528-ccbfb3dbc45e None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:22.390638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:22.420557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70132771-7ec5-4285-b528-ccbfb3dbc45e None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:22.466320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30aef47-0fb2-46e6-b6cf-894d1ee0c758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:22.467061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30aef47-0fb2-46e6-b6cf-894d1ee0c758 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:22.467337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30aef47-0fb2-46e6-b6cf-894d1ee0c758 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:22.467599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30aef47-0fb2-46e6-b6cf-894d1ee0c758 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:22.483639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30aef47-0fb2-46e6-b6cf-894d1ee0c758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-f30aef47-0fb2-46e6-b6cf-894d1ee0c758 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:22.488818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:22.489703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:22.490521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 262/544] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:22.496734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae21267-61ff-4679-9dfd-669d7e70ed32 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:22.605101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae21267-61ff-4679-9dfd-669d7e70ed32 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:22.639651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca2645-79c0-4e77-bd6d-af4f0ad81e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:22.640255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca2645-79c0-4e77-bd6d-af4f0ad81e28 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:22.640530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca2645-79c0-4e77-bd6d-af4f0ad81e28 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:22.640795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca2645-79c0-4e77-bd6d-af4f0ad81e28 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:22.641453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca2645-79c0-4e77-bd6d-af4f0ad81e28 None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:22.641748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca2645-79c0-4e77-bd6d-af4f0ad81e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:22.643145 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca2645-79c0-4e77-bd6d-af4f0ad81e28 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:22.656977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 283/545] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:22.658333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30aef47-0fb2-46e6-b6cf-894d1ee0c758 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:22.707288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a4f31-a12c-4c75-9ebd-20a416818daf None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:22.707596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a4f31-a12c-4c75-9ebd-20a416818daf None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:22.707762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a4f31-a12c-4c75-9ebd-20a416818daf None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:22.709356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 284/546] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:22.715157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be3654-2ed4-4cba-918a-2cd5b4f25b52 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:22.715545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be3654-2ed4-4cba-918a-2cd5b4f25b52 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:22.716221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be3654-2ed4-4cba-918a-2cd5b4f25b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:22.734453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-47be3654-2ed4-4cba-918a-2cd5b4f25b52 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:22.735772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-47be3654-2ed4-4cba-918a-2cd5b4f25b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:22.776629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30aef47-0fb2-46e6-b6cf-894d1ee0c758 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:22.786684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:22.810080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 285/547] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:22.829793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be3654-2ed4-4cba-918a-2cd5b4f25b52 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:22.864812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9da04b-645e-4d92-bfd5-eba0bb8e9e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:22.865555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9da04b-645e-4d92-bfd5-eba0bb8e9e95 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:22.865928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9da04b-645e-4d92-bfd5-eba0bb8e9e95 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:22.866776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9da04b-645e-4d92-bfd5-eba0bb8e9e95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:22.869140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:22.882756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9da04b-645e-4d92-bfd5-eba0bb8e9e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:22.884543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9da04b-645e-4d92-bfd5-eba0bb8e9e95 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-4f9da04b-645e-4d92-bfd5-eba0bb8e9e95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:22.901217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:22.902165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:22.902996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:22.902996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:22.902996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:22.905216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 263/548] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:22.906617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee5326-2d84-40b3-95d2-46009cf22c40 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:22.907100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee5326-2d84-40b3-95d2-46009cf22c40 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:22.907565 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee5326-2d84-40b3-95d2-46009cf22c40 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:22.913612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 264/549] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:22.915211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df6fe2-cac7-4f3d-9015-02fca9ebda8a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:22.915710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df6fe2-cac7-4f3d-9015-02fca9ebda8a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:22.916268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0df6fe2-cac7-4f3d-9015-02fca9ebda8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:22.936138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-e0df6fe2-cac7-4f3d-9015-02fca9ebda8a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:22.937640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e0df6fe2-cac7-4f3d-9015-02fca9ebda8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:22.938232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be3654-2ed4-4cba-918a-2cd5b4f25b52 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:22.973320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509b0bc-5051-4379-b302-0a0192d732e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:22.973942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b0bc-5051-4379-b302-0a0192d732e6 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:22.974243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b0bc-5051-4379-b302-0a0192d732e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:22.974508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b0bc-5051-4379-b302-0a0192d732e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:22.996797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509b0bc-5051-4379-b302-0a0192d732e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-1509b0bc-5051-4379-b302-0a0192d732e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:23.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:23.004310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:23.005605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:23.009618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 286/550] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:23.010687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509b0bc-5051-4379-b302-0a0192d732e6 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.026570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 265/551] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:23.026570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df6fe2-cac7-4f3d-9015-02fca9ebda8a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.071860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:23.149936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0df6fe2-cac7-4f3d-9015-02fca9ebda8a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:23.160056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509b0bc-5051-4379-b302-0a0192d732e6 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:23.172602 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:23.192706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a54011-7aab-48ed-a82b-6586310d3276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:23.193314 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a54011-7aab-48ed-a82b-6586310d3276 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:23.193593 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a54011-7aab-48ed-a82b-6586310d3276 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:23.193861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a54011-7aab-48ed-a82b-6586310d3276 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:23.194587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a54011-7aab-48ed-a82b-6586310d3276 None admin] RBAC: Adding query filter params (name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:23.194885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a54011-7aab-48ed-a82b-6586310d3276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:23.196508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a54011-7aab-48ed-a82b-6586310d3276 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:23.205940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 266/552] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:23.206329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4233b-d211-4e9a-820f-f92768710acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:23.206944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a54011-7aab-48ed-a82b-6586310d3276 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.207860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4233b-d211-4e9a-820f-f92768710acf None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:23.208282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4233b-d211-4e9a-820f-f92768710acf None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:23.208706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4233b-d211-4e9a-820f-f92768710acf None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:23.210262 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4233b-d211-4e9a-820f-f92768710acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:23.210728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4233b-d211-4e9a-820f-f92768710acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:23.212048 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac4233b-d211-4e9a-820f-f92768710acf None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:23.233014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 287/553] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:23.233014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4233b-d211-4e9a-820f-f92768710acf None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.334271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:23.337325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a54011-7aab-48ed-a82b-6586310d3276 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:23.343499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac4233b-d211-4e9a-820f-f92768710acf None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:23.375116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef441002-832e-49b0-9f20-759d4d8d653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:23.376018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441002-832e-49b0-9f20-759d4d8d653f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:23.376538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441002-832e-49b0-9f20-759d4d8d653f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:23.376965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58036d6d-4ad4-4c82-961e-ec0c699ded88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:23.377191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef441002-832e-49b0-9f20-759d4d8d653f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:23.377771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58036d6d-4ad4-4c82-961e-ec0c699ded88 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:23.378153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58036d6d-4ad4-4c82-961e-ec0c699ded88 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:23.378495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58036d6d-4ad4-4c82-961e-ec0c699ded88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:23.389833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58036d6d-4ad4-4c82-961e-ec0c699ded88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:23.390673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef441002-832e-49b0-9f20-759d4d8d653f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:23.391471 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58036d6d-4ad4-4c82-961e-ec0c699ded88 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-ef441002-832e-49b0-9f20-759d4d8d653f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:23.398402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:23.399291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:23.400095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:23.403214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 267/554] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:23.404584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef441002-832e-49b0-9f20-759d4d8d653f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-58036d6d-4ad4-4c82-961e-ec0c699ded88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:23.409541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:23.410643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:23.411706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:23.411706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:23.411706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:23.413478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 288/555] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:23.414316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58036d6d-4ad4-4c82-961e-ec0c699ded88 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.456401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:23.522107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58036d6d-4ad4-4c82-961e-ec0c699ded88 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:23.522863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef441002-832e-49b0-9f20-759d4d8d653f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:23.560138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a297dd-c399-41b6-a248-c4c24e7055a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:23.560394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06de4e-c21b-48db-89ef-04a8c4a4c727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:23.560983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06de4e-c21b-48db-89ef-04a8c4a4c727 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:23.561303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a297dd-c399-41b6-a248-c4c24e7055a7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:23.561424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06de4e-c21b-48db-89ef-04a8c4a4c727 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:23.561810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06de4e-c21b-48db-89ef-04a8c4a4c727 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:23.561995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a297dd-c399-41b6-a248-c4c24e7055a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:23.562396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a297dd-c399-41b6-a248-c4c24e7055a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:23.562769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06de4e-c21b-48db-89ef-04a8c4a4c727 None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:23.563164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06de4e-c21b-48db-89ef-04a8c4a4c727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:23.564741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06de4e-c21b-48db-89ef-04a8c4a4c727 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:23.571688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a297dd-c399-41b6-a248-c4c24e7055a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:23.573271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a297dd-c399-41b6-a248-c4c24e7055a7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:23.574542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 289/556] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:23.575708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06de4e-c21b-48db-89ef-04a8c4a4c727 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-d8a297dd-c399-41b6-a248-c4c24e7055a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:23.587381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:23.588332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:23.589692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:23.589692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:23.589692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:23.589692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:23.589692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:23.589692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:23.589692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:23.589692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:23.589692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:23.589692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:23.592108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:23.597368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 268/557] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:23.598448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a297dd-c399-41b6-a248-c4c24e7055a7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.694637 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06de4e-c21b-48db-89ef-04a8c4a4c727 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:23.703619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:23.707829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a297dd-c399-41b6-a248-c4c24e7055a7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:23.732115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8818351-d4d9-4762-98a6-caeb8dde3a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:23.733656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8818351-d4d9-4762-98a6-caeb8dde3a44 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:23.734304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8818351-d4d9-4762-98a6-caeb8dde3a44 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:23.734660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8818351-d4d9-4762-98a6-caeb8dde3a44 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:23.735379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8818351-d4d9-4762-98a6-caeb8dde3a44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:23.735746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8818351-d4d9-4762-98a6-caeb8dde3a44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:23.737287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8818351-d4d9-4762-98a6-caeb8dde3a44 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:23.746662 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3545cc-67ff-4ab7-97d6-4c83f7dbd34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:23.747572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3545cc-67ff-4ab7-97d6-4c83f7dbd34a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:23.748024 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3545cc-67ff-4ab7-97d6-4c83f7dbd34a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:23.748563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3545cc-67ff-4ab7-97d6-4c83f7dbd34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:23.752385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 290/558] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:23.753362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8818351-d4d9-4762-98a6-caeb8dde3a44 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.761054 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3545cc-67ff-4ab7-97d6-4c83f7dbd34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:23.762902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3545cc-67ff-4ab7-97d6-4c83f7dbd34a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-1e3545cc-67ff-4ab7-97d6-4c83f7dbd34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:23.778426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:23.779493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:23.780340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:23.780340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:23.780340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:23.782648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 269/559] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:23.783797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3545cc-67ff-4ab7-97d6-4c83f7dbd34a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.827178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:23.860765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8818351-d4d9-4762-98a6-caeb8dde3a44 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:23.892959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3545cc-67ff-4ab7-97d6-4c83f7dbd34a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:23.898607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ed4df-f32d-45b9-ae63-95d8d1a1442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:23.899218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ed4df-f32d-45b9-ae63-95d8d1a1442a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:23.899558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ed4df-f32d-45b9-ae63-95d8d1a1442a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:23.899832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ed4df-f32d-45b9-ae63-95d8d1a1442a None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:23.900563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ed4df-f32d-45b9-ae63-95d8d1a1442a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:23.900866 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ed4df-f32d-45b9-ae63-95d8d1a1442a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:23.902330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ed4df-f32d-45b9-ae63-95d8d1a1442a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:23.926574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 291/560] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:23.926574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ed4df-f32d-45b9-ae63-95d8d1a1442a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.937891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30bef2-0407-4003-9238-bde0598858ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:23.938972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30bef2-0407-4003-9238-bde0598858ac None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:23.939546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30bef2-0407-4003-9238-bde0598858ac None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:23.940011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30bef2-0407-4003-9238-bde0598858ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:23.948492 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30bef2-0407-4003-9238-bde0598858ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:23.950395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30bef2-0407-4003-9238-bde0598858ac None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-ed30bef2-0407-4003-9238-bde0598858ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:23.962998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:23.963925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:23.964920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:23.964920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:23.964920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:23.964920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:23.964920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:23.964920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:23.964920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:23.964920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:23.964920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:23.964920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:23.967779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 270/561] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:23.968859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30bef2-0407-4003-9238-bde0598858ac None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:23.983073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:24.032764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ed4df-f32d-45b9-ae63-95d8d1a1442a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:24.062356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:24.070546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879f50-2cd3-4b1c-af3a-7214f02846e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:24.071138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879f50-2cd3-4b1c-af3a-7214f02846e1 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.075283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879f50-2cd3-4b1c-af3a-7214f02846e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.075283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe879f50-2cd3-4b1c-af3a-7214f02846e1 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.075283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe879f50-2cd3-4b1c-af3a-7214f02846e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:24.075283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe879f50-2cd3-4b1c-af3a-7214f02846e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:24.075283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe879f50-2cd3-4b1c-af3a-7214f02846e1 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:24.079446 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30bef2-0407-4003-9238-bde0598858ac None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:24.087173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 292/562] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:24.088161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879f50-2cd3-4b1c-af3a-7214f02846e1 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:24.117058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4615b-420b-4e51-b0be-578d33404201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:24.118031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4615b-420b-4e51-b0be-578d33404201 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.118528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4615b-420b-4e51-b0be-578d33404201 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.118941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4615b-420b-4e51-b0be-578d33404201 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.188619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe879f50-2cd3-4b1c-af3a-7214f02846e1 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:24.190157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4615b-420b-4e51-b0be-578d33404201 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:24.191616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4615b-420b-4e51-b0be-578d33404201 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:24.202527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:24.221837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328caa9-9075-42cd-ad22-1f78e72c541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:24.222608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328caa9-9075-42cd-ad22-1f78e72c541b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.222781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328caa9-9075-42cd-ad22-1f78e72c541b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.223034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328caa9-9075-42cd-ad22-1f78e72c541b None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.223741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328caa9-9075-42cd-ad22-1f78e72c541b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:24.224073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328caa9-9075-42cd-ad22-1f78e72c541b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:24.226379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328caa9-9075-42cd-ad22-1f78e72c541b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:24.243952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 293/563] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Aug 9 16:32:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:24.245155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d40694-efcc-4092-80de-225213abf34e None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.245439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d40694-efcc-4092-80de-225213abf34e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.245709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d40694-efcc-4092-80de-225213abf34e None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.246972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 294/564] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:24.251114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d40694-efcc-4092-80de-225213abf34e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:24.291734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 271/565] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:24.292796 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393b692-9cb0-4834-a561-dac6f252122e None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.293078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393b692-9cb0-4834-a561-dac6f252122e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.293348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393b692-9cb0-4834-a561-dac6f252122e None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.294980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 272/566] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:24.295574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393b692-9cb0-4834-a561-dac6f252122e None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:24.303075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:24.355799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d40694-efcc-4092-80de-225213abf34e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:24.388499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed9307-4c6e-4e26-a897-e178e3c95e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:24.389107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed9307-4c6e-4e26-a897-e178e3c95e88 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.389380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed9307-4c6e-4e26-a897-e178e3c95e88 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.389892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed9307-4c6e-4e26-a897-e178e3c95e88 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.395474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a393b692-9cb0-4834-a561-dac6f252122e None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:24.397159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed9307-4c6e-4e26-a897-e178e3c95e88 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:24.399255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed9307-4c6e-4e26-a897-e178e3c95e88 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:24.406571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-faed9307-4c6e-4e26-a897-e178e3c95e88 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:24.410698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:24.411674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:24.412704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 295/567] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:24.413281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05c454-7e4d-4e2e-8fd1-e3ea7f45aa52 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.413281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05c454-7e4d-4e2e-8fd1-e3ea7f45aa52 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.413815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05c454-7e4d-4e2e-8fd1-e3ea7f45aa52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.431706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5a05c454-7e4d-4e2e-8fd1-e3ea7f45aa52 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:24.432896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5a05c454-7e4d-4e2e-8fd1-e3ea7f45aa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:24.433418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759659a3-949c-4e49-a5a2-0badc66be5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:24.434058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759659a3-949c-4e49-a5a2-0badc66be5db None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.434333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759659a3-949c-4e49-a5a2-0badc66be5db None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.434597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759659a3-949c-4e49-a5a2-0badc66be5db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.445344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759659a3-949c-4e49-a5a2-0badc66be5db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:24.446811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759659a3-949c-4e49-a5a2-0badc66be5db None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-759659a3-949c-4e49-a5a2-0badc66be5db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:24.461574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:24.462646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:24.463541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:24.463541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:24.463541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:24.463541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 273/568] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:24.463541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759659a3-949c-4e49-a5a2-0badc66be5db None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:24.503566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 296/569] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:24.505692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b24508-b13f-465d-ac18-1869d354676b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.506149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b24508-b13f-465d-ac18-1869d354676b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.506566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b24508-b13f-465d-ac18-1869d354676b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.533021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-09b24508-b13f-465d-ac18-1869d354676b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:24.534240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-09b24508-b13f-465d-ac18-1869d354676b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:24.563734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:24.572002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759659a3-949c-4e49-a5a2-0badc66be5db None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:24.601042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 297/570] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:24.601899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b24508-b13f-465d-ac18-1869d354676b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:24.606205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0cb4-341e-4739-be3b-89c32dd55795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:24.606829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0cb4-341e-4739-be3b-89c32dd55795 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.607101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0cb4-341e-4739-be3b-89c32dd55795 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.607368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b0cb4-341e-4739-be3b-89c32dd55795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.618838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0cb4-341e-4739-be3b-89c32dd55795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:24.620555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b0cb4-341e-4739-be3b-89c32dd55795 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-529b0cb4-341e-4739-be3b-89c32dd55795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:24.635498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:24.641280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:24.642278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:24.642278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:24.642278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:24.642278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 274/571] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:24.642278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0cb4-341e-4739-be3b-89c32dd55795 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:24.655935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:24.699404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b24508-b13f-465d-ac18-1869d354676b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:24.733652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a18ee-24d2-4f28-9779-226b0c956481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:24.734275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a18ee-24d2-4f28-9779-226b0c956481 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.734551 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a18ee-24d2-4f28-9779-226b0c956481 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.734915 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a18ee-24d2-4f28-9779-226b0c956481 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.735632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a18ee-24d2-4f28-9779-226b0c956481 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:24.735931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a18ee-24d2-4f28-9779-226b0c956481 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:24.737345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a18ee-24d2-4f28-9779-226b0c956481 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:24.738395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b0cb4-341e-4739-be3b-89c32dd55795 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:24.754514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 298/572] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Aug 9 16:32:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:24.755523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a18ee-24d2-4f28-9779-226b0c956481 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:24.772286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fd25c-e109-4f43-b737-4ef2b59e2227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:24.772871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fd25c-e109-4f43-b737-4ef2b59e2227 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.773144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fd25c-e109-4f43-b737-4ef2b59e2227 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.773405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fd25c-e109-4f43-b737-4ef2b59e2227 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.774053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fd25c-e109-4f43-b737-4ef2b59e2227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:24.774344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fd25c-e109-4f43-b737-4ef2b59e2227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:24.775681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fd25c-e109-4f43-b737-4ef2b59e2227 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:24.780638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:24.787275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 275/573] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:24.787923 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fd25c-e109-4f43-b737-4ef2b59e2227 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:24.855083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a18ee-24d2-4f28-9779-226b0c956481 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:24.864877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:24.888497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90f8b0-c78b-4f5a-bdcd-4dbe858ce36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:24.889102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90f8b0-c78b-4f5a-bdcd-4dbe858ce36f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.889373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90f8b0-c78b-4f5a-bdcd-4dbe858ce36f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.889636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90f8b0-c78b-4f5a-bdcd-4dbe858ce36f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.889755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fd25c-e109-4f43-b737-4ef2b59e2227 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:24.900297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90f8b0-c78b-4f5a-bdcd-4dbe858ce36f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-2d90f8b0-c78b-4f5a-bdcd-4dbe858ce36f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:24.905068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:24.906040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:24.907038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 299/574] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:24.911784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90f8b0-c78b-4f5a-bdcd-4dbe858ce36f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:24.930947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6239fc-00f1-44e0-874f-8c1879baa4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:24.931585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6239fc-00f1-44e0-874f-8c1879baa4db None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:24.931865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6239fc-00f1-44e0-874f-8c1879baa4db None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:24.932134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6239fc-00f1-44e0-874f-8c1879baa4db None admin] PATH_INFO: `/v3/roles/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:24.945854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6239fc-00f1-44e0-874f-8c1879baa4db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-7e6239fc-00f1-44e0-874f-8c1879baa4db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:24.950462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:24.951588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 276/575] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:24.952802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6239fc-00f1-44e0-874f-8c1879baa4db None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.000102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:25.015321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90f8b0-c78b-4f5a-bdcd-4dbe858ce36f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.049178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc553f6-450e-4b61-ae3f-0b47cfe1f9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.049769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc553f6-450e-4b61-ae3f-0b47cfe1f9f4 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.050139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc553f6-450e-4b61-ae3f-0b47cfe1f9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.050410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc553f6-450e-4b61-ae3f-0b47cfe1f9f4 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.051042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc553f6-450e-4b61-ae3f-0b47cfe1f9f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:25.051286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6239fc-00f1-44e0-874f-8c1879baa4db None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.051381 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc553f6-450e-4b61-ae3f-0b47cfe1f9f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.052685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc553f6-450e-4b61-ae3f-0b47cfe1f9f4 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.064342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 300/576] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:25.065125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc553f6-450e-4b61-ae3f-0b47cfe1f9f4 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.089824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b429d21-2d42-4a13-a72e-a9fcccb27629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.090477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b429d21-2d42-4a13-a72e-a9fcccb27629 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.091004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b429d21-2d42-4a13-a72e-a9fcccb27629 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.091270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b429d21-2d42-4a13-a72e-a9fcccb27629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.093173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:25.102255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b429d21-2d42-4a13-a72e-a9fcccb27629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.103682 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b429d21-2d42-4a13-a72e-a9fcccb27629 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-9b429d21-2d42-4a13-a72e-a9fcccb27629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:25.118357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:25.119461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:25.120543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:25.120543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:25.120543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:25.120543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 277/577] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:25.120543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b429d21-2d42-4a13-a72e-a9fcccb27629 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.166578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc553f6-450e-4b61-ae3f-0b47cfe1f9f4 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.199952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9153a6c-e1b4-4509-866e-c1f3ecefec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.200564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9153a6c-e1b4-4509-866e-c1f3ecefec4b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.200835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9153a6c-e1b4-4509-866e-c1f3ecefec4b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.201101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9153a6c-e1b4-4509-866e-c1f3ecefec4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.207732 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9153a6c-e1b4-4509-866e-c1f3ecefec4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.209159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9153a6c-e1b4-4509-866e-c1f3ecefec4b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.218324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b429d21-2d42-4a13-a72e-a9fcccb27629 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-b9153a6c-e1b4-4509-866e-c1f3ecefec4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:25.220030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:25.221034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:25.221924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:25.221924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:25.221924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:25.221924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:25.221924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:25.221924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:25.221924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:25.221924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:25.221924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 301/578] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:25.221924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9153a6c-e1b4-4509-866e-c1f3ecefec4b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.226319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:25.259147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573bdbd-6372-4704-be4e-7cfddd1aaa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.259748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573bdbd-6372-4704-be4e-7cfddd1aaa79 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.260027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573bdbd-6372-4704-be4e-7cfddd1aaa79 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.260298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573bdbd-6372-4704-be4e-7cfddd1aaa79 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.260949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573bdbd-6372-4704-be4e-7cfddd1aaa79 None admin] RBAC: Adding query filter params (name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:25.261249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573bdbd-6372-4704-be4e-7cfddd1aaa79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.262632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573bdbd-6372-4704-be4e-7cfddd1aaa79 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.270619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 278/579] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:25.271401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573bdbd-6372-4704-be4e-7cfddd1aaa79 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.322922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9153a6c-e1b4-4509-866e-c1f3ecefec4b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.325709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:25.359076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765ed86-2bfc-4b3b-a3bc-0de63e0cc57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.359735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765ed86-2bfc-4b3b-a3bc-0de63e0cc57a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.360145 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765ed86-2bfc-4b3b-a3bc-0de63e0cc57a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.360284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765ed86-2bfc-4b3b-a3bc-0de63e0cc57a None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.360944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765ed86-2bfc-4b3b-a3bc-0de63e0cc57a None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:25.361247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765ed86-2bfc-4b3b-a3bc-0de63e0cc57a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.362626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765ed86-2bfc-4b3b-a3bc-0de63e0cc57a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.370602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 302/580] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:25.371363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765ed86-2bfc-4b3b-a3bc-0de63e0cc57a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.386150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573bdbd-6372-4704-be4e-7cfddd1aaa79 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.419789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2c51a-856d-4ce2-816d-5ce14a7b6221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.420480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2c51a-856d-4ce2-816d-5ce14a7b6221 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.420820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2c51a-856d-4ce2-816d-5ce14a7b6221 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.421090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2c51a-856d-4ce2-816d-5ce14a7b6221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.427699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2c51a-856d-4ce2-816d-5ce14a7b6221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.429623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2c51a-856d-4ce2-816d-5ce14a7b6221 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.436560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-cfe2c51a-856d-4ce2-816d-5ce14a7b6221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:25.440296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:25.441344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:25.442719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:25.442719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:25.442719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:25.442719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:25.442719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:25.442719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:25.442719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:25.442719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:25.442719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 279/581] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:25.442719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2c51a-856d-4ce2-816d-5ce14a7b6221 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.474131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765ed86-2bfc-4b3b-a3bc-0de63e0cc57a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.511435 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb3c1-21d5-41cf-a54a-0ef78aa8d310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.512039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb3c1-21d5-41cf-a54a-0ef78aa8d310 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.512319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb3c1-21d5-41cf-a54a-0ef78aa8d310 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.512589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44eb3c1-21d5-41cf-a54a-0ef78aa8d310 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.513228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44eb3c1-21d5-41cf-a54a-0ef78aa8d310 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:25.513526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44eb3c1-21d5-41cf-a54a-0ef78aa8d310 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.514905 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44eb3c1-21d5-41cf-a54a-0ef78aa8d310 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.526581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 303/582] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:25.527385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb3c1-21d5-41cf-a54a-0ef78aa8d310 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.546315 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2c51a-856d-4ce2-816d-5ce14a7b6221 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.555667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:25.581039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdcb0a-37fc-4420-858f-a36417cf9378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.581673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdcb0a-37fc-4420-858f-a36417cf9378 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.582104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdcb0a-37fc-4420-858f-a36417cf9378 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.582372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdcb0a-37fc-4420-858f-a36417cf9378 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.583053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdcb0a-37fc-4420-858f-a36417cf9378 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:25.583348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdcb0a-37fc-4420-858f-a36417cf9378 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.585269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdcb0a-37fc-4420-858f-a36417cf9378 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.597189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 280/583] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:25.598185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdcb0a-37fc-4420-858f-a36417cf9378 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.627195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44eb3c1-21d5-41cf-a54a-0ef78aa8d310 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.652877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:25.661111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114b82c-0ae0-419a-9ad5-e8a6882419f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.661724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114b82c-0ae0-419a-9ad5-e8a6882419f3 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.662030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114b82c-0ae0-419a-9ad5-e8a6882419f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.662304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114b82c-0ae0-419a-9ad5-e8a6882419f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.672467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114b82c-0ae0-419a-9ad5-e8a6882419f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.673893 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114b82c-0ae0-419a-9ad5-e8a6882419f3 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-d114b82c-0ae0-419a-9ad5-e8a6882419f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:25.688280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:25.689420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:25.690533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:25.690533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:25.690533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:25.694022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 304/584] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:25.694766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114b82c-0ae0-419a-9ad5-e8a6882419f3 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.695577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdcb0a-37fc-4420-858f-a36417cf9378 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.730578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749940e2-f5d8-4ce5-926a-831477541886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.731198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749940e2-f5d8-4ce5-926a-831477541886 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.731477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749940e2-f5d8-4ce5-926a-831477541886 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.731735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749940e2-f5d8-4ce5-926a-831477541886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.741553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749940e2-f5d8-4ce5-926a-831477541886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.743009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749940e2-f5d8-4ce5-926a-831477541886 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-749940e2-f5d8-4ce5-926a-831477541886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:25.756860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:25.757762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:25.758678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:25.758678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:25.758678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:25.758678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 281/585] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:25.761774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749940e2-f5d8-4ce5-926a-831477541886 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.783321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:25.793070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114b82c-0ae0-419a-9ad5-e8a6882419f3 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.825737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26c527-72b5-41d5-bd1c-0d2a8763498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.826373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26c527-72b5-41d5-bd1c-0d2a8763498f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.826657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26c527-72b5-41d5-bd1c-0d2a8763498f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.826936 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26c527-72b5-41d5-bd1c-0d2a8763498f None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.827612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26c527-72b5-41d5-bd1c-0d2a8763498f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:25.827913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26c527-72b5-41d5-bd1c-0d2a8763498f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.829338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26c527-72b5-41d5-bd1c-0d2a8763498f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.847145 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 305/586] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:25.848350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26c527-72b5-41d5-bd1c-0d2a8763498f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.859267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749940e2-f5d8-4ce5-926a-831477541886 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.890034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:25.891684 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6704ae-73f3-4534-b9b2-cce9482a99d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.892295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6704ae-73f3-4534-b9b2-cce9482a99d6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.892568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6704ae-73f3-4534-b9b2-cce9482a99d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.892832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6704ae-73f3-4534-b9b2-cce9482a99d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.899814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6704ae-73f3-4534-b9b2-cce9482a99d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:25.901230 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6704ae-73f3-4534-b9b2-cce9482a99d6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-6f6704ae-73f3-4534-b9b2-cce9482a99d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:25.912049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:25.913195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:25.914306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:25.914306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:25.914306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:25.914306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:25.914306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:25.914306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:25.914306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:25.914306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:25.914306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 282/587] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:25.914875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6704ae-73f3-4534-b9b2-cce9482a99d6 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:25.950960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26c527-72b5-41d5-bd1c-0d2a8763498f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:25.985236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe7ee1-cadc-490b-8527-7a834b2cfb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:25.985880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7ee1-cadc-490b-8527-7a834b2cfb7b None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:25.986202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7ee1-cadc-490b-8527-7a834b2cfb7b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:25.986472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7ee1-cadc-490b-8527-7a834b2cfb7b None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/5d17aa4b23e04290b6c8b1924b0dcf32/roles/844d711adf9b4c1f8471a01e6031e1e6` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:25.995633 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:26.001046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe7ee1-cadc-490b-8527-7a834b2cfb7b None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=5d17aa4b23e04290b6c8b1924b0dcf32, role_id=844d711adf9b4c1f8471a01e6031e1e6)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:26.004218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe7ee1-cadc-490b-8527-7a834b2cfb7b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:26.014515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6704ae-73f3-4534-b9b2-cce9482a99d6 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:26.047333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebb623-c278-4277-af72-e688aa8518ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:26.047918 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebb623-c278-4277-af72-e688aa8518ad None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:26.048192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebb623-c278-4277-af72-e688aa8518ad None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:26.048451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebb623-c278-4277-af72-e688aa8518ad None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:26.049065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebb623-c278-4277-af72-e688aa8518ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:26.049357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebb623-c278-4277-af72-e688aa8518ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:26.050748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebb623-c278-4277-af72-e688aa8518ad None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:26.062321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 283/588] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:26.063128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebb623-c278-4277-af72-e688aa8518ad None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:26.114842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 306/589] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:32:25 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/5d17aa4b23e04290b6c8b1924b0dcf32/roles/844d711adf9b4c1f8471a01e6031e1e6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:26.115777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe7ee1-cadc-490b-8527-7a834b2cfb7b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:26.172926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebb623-c278-4277-af72-e688aa8518ad None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:26.214070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570c3d3-f938-4501-ab79-39c773cc0043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:26.214680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570c3d3-f938-4501-ab79-39c773cc0043 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:26.214955 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570c3d3-f938-4501-ab79-39c773cc0043 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:26.215219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570c3d3-f938-4501-ab79-39c773cc0043 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:26.216109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570c3d3-f938-4501-ab79-39c773cc0043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:26.216437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570c3d3-f938-4501-ab79-39c773cc0043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:26.217815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570c3d3-f938-4501-ab79-39c773cc0043 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:26.228141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:26.231524 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:26.232202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 284/590] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:26.237246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570c3d3-f938-4501-ab79-39c773cc0043 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:26.250658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe7ee1-cadc-490b-8527-7a834b2cfb7b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:26.294383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35905d3e-e899-46b9-8fee-931c58730c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:26.295020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35905d3e-e899-46b9-8fee-931c58730c49 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:26.295333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35905d3e-e899-46b9-8fee-931c58730c49 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:26.295640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35905d3e-e899-46b9-8fee-931c58730c49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:26.296472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35905d3e-e899-46b9-8fee-931c58730c49 None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=35371b488c474b7f851af16bdad34f69) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:26.296830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35905d3e-e899-46b9-8fee-931c58730c49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:26.298435 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35905d3e-e899-46b9-8fee-931c58730c49 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:26.306665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 307/591] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:26 2022] GET /identity/v3/role_assignments?user.id=35371b488c474b7f851af16bdad34f69&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:26.307858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35905d3e-e899-46b9-8fee-931c58730c49 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:26.353966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570c3d3-f938-4501-ab79-39c773cc0043 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:26.396173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809ad99-0d6f-45a3-b3b2-7a707f9a296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:26.396782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809ad99-0d6f-45a3-b3b2-7a707f9a296c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:26.397056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809ad99-0d6f-45a3-b3b2-7a707f9a296c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:26.397365 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809ad99-0d6f-45a3-b3b2-7a707f9a296c None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:26.398111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809ad99-0d6f-45a3-b3b2-7a707f9a296c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:26.398395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809ad99-0d6f-45a3-b3b2-7a707f9a296c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:26.399938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809ad99-0d6f-45a3-b3b2-7a707f9a296c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:26.413980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 285/592] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:26.414922 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809ad99-0d6f-45a3-b3b2-7a707f9a296c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:26.458314 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35905d3e-e899-46b9-8fee-931c58730c49 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:26.483648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:26.503394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde2e16-7ad2-4798-baee-231d37ee8efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:26.513615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2e16-7ad2-4798-baee-231d37ee8efa None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:26.513615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2e16-7ad2-4798-baee-231d37ee8efa None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:26.513615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2e16-7ad2-4798-baee-231d37ee8efa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:26.518830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde2e16-7ad2-4798-baee-231d37ee8efa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:26.520297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde2e16-7ad2-4798-baee-231d37ee8efa None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:26.537580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809ad99-0d6f-45a3-b3b2-7a707f9a296c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-6cde2e16-7ad2-4798-baee-231d37ee8efa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:26.543645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:26.544629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:26.545747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:26.545747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:26.545747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:26.545747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:26.545747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:26.545747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:26.545747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:26.545747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:26.545747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 16:32:26.545747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:26.547919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 308/593] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:26.548974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde2e16-7ad2-4798-baee-231d37ee8efa None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:26.569342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:26.585030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a434d33-7a9d-49ce-b197-709d5073880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:26.585817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434d33-7a9d-49ce-b197-709d5073880e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:26.586207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434d33-7a9d-49ce-b197-709d5073880e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:26.586570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434d33-7a9d-49ce-b197-709d5073880e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:26.597978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a434d33-7a9d-49ce-b197-709d5073880e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:26.597978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a434d33-7a9d-49ce-b197-709d5073880e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-5a434d33-7a9d-49ce-b197-709d5073880e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:26.610194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:26.611326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:26.612453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:26.612453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:26.612453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:26.612453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:26.612453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:26.612453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:26.612453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:26.612453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:26.612453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 16:32:26.612453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:26.618313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 286/594] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:26.619239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a434d33-7a9d-49ce-b197-709d5073880e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:26.678347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde2e16-7ad2-4798-baee-231d37ee8efa None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:26.713969 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be906e-e50e-4740-a7de-6bf8c2719070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:26.714891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be906e-e50e-4740-a7de-6bf8c2719070 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:26.715075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be906e-e50e-4740-a7de-6bf8c2719070 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:26.715435 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be906e-e50e-4740-a7de-6bf8c2719070 None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/33003a9bf4454f5bac2e6ad73544efc4/roles/844d711adf9b4c1f8471a01e6031e1e6` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:26.733052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be906e-e50e-4740-a7de-6bf8c2719070 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=33003a9bf4454f5bac2e6ad73544efc4, role_id=844d711adf9b4c1f8471a01e6031e1e6)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:26.734789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be906e-e50e-4740-a7de-6bf8c2719070 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:26.747639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a434d33-7a9d-49ce-b197-709d5073880e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:26.769751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:26.785099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee314c-beea-4fa0-9254-6d9449f36771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:26.785706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee314c-beea-4fa0-9254-6d9449f36771 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:26.786008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee314c-beea-4fa0-9254-6d9449f36771 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:26.786279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee314c-beea-4fa0-9254-6d9449f36771 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:26.787002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee314c-beea-4fa0-9254-6d9449f36771 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:26.787301 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee314c-beea-4fa0-9254-6d9449f36771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:26.789230 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee314c-beea-4fa0-9254-6d9449f36771 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:26.813034 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 287/595] 10.222.0.79 () {56 vars in 1139 bytes} [Tue Aug 9 16:32:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:26.813034 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee314c-beea-4fa0-9254-6d9449f36771 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:26.817189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 309/596] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:32:26 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/33003a9bf4454f5bac2e6ad73544efc4/roles/844d711adf9b4c1f8471a01e6031e1e6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:26.818345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be906e-e50e-4740-a7de-6bf8c2719070 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:26.960669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:27.023875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee314c-beea-4fa0-9254-6d9449f36771 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:27.031457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be906e-e50e-4740-a7de-6bf8c2719070 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:27.062760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1b3db-0996-4504-8361-a3700ebb64db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:27.063731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1b3db-0996-4504-8361-a3700ebb64db None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:27.064210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1b3db-0996-4504-8361-a3700ebb64db None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:27.064679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1b3db-0996-4504-8361-a3700ebb64db None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:27.065594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1b3db-0996-4504-8361-a3700ebb64db None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:27.066133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1b3db-0996-4504-8361-a3700ebb64db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:27.067778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1b3db-0996-4504-8361-a3700ebb64db None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:27.072070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf2eba-0cac-4d85-b185-58a46ce90be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:27.072700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf2eba-0cac-4d85-b185-58a46ce90be6 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:27.072979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf2eba-0cac-4d85-b185-58a46ce90be6 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:27.073246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf2eba-0cac-4d85-b185-58a46ce90be6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:27.087710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf2eba-0cac-4d85-b185-58a46ce90be6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:27.089354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf2eba-0cac-4d85-b185-58a46ce90be6 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:27.089589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 288/597] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Aug 9 16:32:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:27.095447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1b3db-0996-4504-8361-a3700ebb64db None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-d5bf2eba-0cac-4d85-b185-58a46ce90be6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:27.108803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:27.109785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:27.110796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:27.110796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:27.110796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:27.110796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 310/598] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:27.116418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf2eba-0cac-4d85-b185-58a46ce90be6 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:27.134846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:27.261456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1b3db-0996-4504-8361-a3700ebb64db None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:27.288859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf2eba-0cac-4d85-b185-58a46ce90be6 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:27.308639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:27.334230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc567a2-defb-4608-8d1c-3e2b195cd8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:27.335617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc567a2-defb-4608-8d1c-3e2b195cd8f9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:27.336041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc567a2-defb-4608-8d1c-3e2b195cd8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:27.336478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc567a2-defb-4608-8d1c-3e2b195cd8f9 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:27.337272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc567a2-defb-4608-8d1c-3e2b195cd8f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:27.337713 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc567a2-defb-4608-8d1c-3e2b195cd8f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:27.340570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc567a2-defb-4608-8d1c-3e2b195cd8f9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:27.355984 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a853cd-2c8d-4e5e-b84e-46d31fe71fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:27.356611 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a853cd-2c8d-4e5e-b84e-46d31fe71fd4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:27.357043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a853cd-2c8d-4e5e-b84e-46d31fe71fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:27.357222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a853cd-2c8d-4e5e-b84e-46d31fe71fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:27.357672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 311/599] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:27.363894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc567a2-defb-4608-8d1c-3e2b195cd8f9 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:27.380653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a853cd-2c8d-4e5e-b84e-46d31fe71fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:27.383770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a853cd-2c8d-4e5e-b84e-46d31fe71fd4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-60a853cd-2c8d-4e5e-b84e-46d31fe71fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:27.410442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:27.411608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:27.412686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:27.412686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:27.412686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:27.415474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 289/600] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:27.420555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a853cd-2c8d-4e5e-b84e-46d31fe71fd4 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:27.509591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc567a2-defb-4608-8d1c-3e2b195cd8f9 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:27.560961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd5be7-3616-4570-9951-0e2418dcbf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:27.561942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5be7-3616-4570-9951-0e2418dcbf6c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:27.562383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5be7-3616-4570-9951-0e2418dcbf6c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:27.562895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd5be7-3616-4570-9951-0e2418dcbf6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:27.570578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd5be7-3616-4570-9951-0e2418dcbf6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:27.577720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd5be7-3616-4570-9951-0e2418dcbf6c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-18bd5be7-3616-4570-9951-0e2418dcbf6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:27.598700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:27.599795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:27.600908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:27.600908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:27.600908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:27.600908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:27.600908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:27.600908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:27.600908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:27.600908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:27.610234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 312/601] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:27.614039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd5be7-3616-4570-9951-0e2418dcbf6c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:27.629032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:27.649114 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a853cd-2c8d-4e5e-b84e-46d31fe71fd4 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:27.682088 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:27.717422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61619019-ffc7-4c5c-ad23-f0f0fe9f7be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:27.718485 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61619019-ffc7-4c5c-ad23-f0f0fe9f7be6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:27.720010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61619019-ffc7-4c5c-ad23-f0f0fe9f7be6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:27.721093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61619019-ffc7-4c5c-ad23-f0f0fe9f7be6 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:27.722540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61619019-ffc7-4c5c-ad23-f0f0fe9f7be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:27.723138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61619019-ffc7-4c5c-ad23-f0f0fe9f7be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:27.726632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61619019-ffc7-4c5c-ad23-f0f0fe9f7be6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:27.749266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 290/602] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:27.753535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61619019-ffc7-4c5c-ad23-f0f0fe9f7be6 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:27.802900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd5be7-3616-4570-9951-0e2418dcbf6c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:27.867055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1129084-59ee-41f4-bab3-79d8bd8d9d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:27.868612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1129084-59ee-41f4-bab3-79d8bd8d9d3a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:27.869572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1129084-59ee-41f4-bab3-79d8bd8d9d3a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:27.870158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1129084-59ee-41f4-bab3-79d8bd8d9d3a None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:27.872196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1129084-59ee-41f4-bab3-79d8bd8d9d3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:27.873975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1129084-59ee-41f4-bab3-79d8bd8d9d3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:27.876601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1129084-59ee-41f4-bab3-79d8bd8d9d3a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:27.899784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 313/603] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:27.910282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1129084-59ee-41f4-bab3-79d8bd8d9d3a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:27.949583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61619019-ffc7-4c5c-ad23-f0f0fe9f7be6 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:28.020160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8b2ed-e7d9-4687-9740-f008c8f9ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:28.021614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8b2ed-e7d9-4687-9740-f008c8f9ca02 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:28.022521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8b2ed-e7d9-4687-9740-f008c8f9ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:28.023088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8b2ed-e7d9-4687-9740-f008c8f9ca02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:28.033469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:28.035789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8b2ed-e7d9-4687-9740-f008c8f9ca02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:28.038737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8b2ed-e7d9-4687-9740-f008c8f9ca02 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:28.045459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1129084-59ee-41f4-bab3-79d8bd8d9d3a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-62e8b2ed-e7d9-4687-9740-f008c8f9ca02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:28.068876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:28.069811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:28.070703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:28.070703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:28.070703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:28.070703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:28.070703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:28.070703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:28.070703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:28.070703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:28.080142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 291/604] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:28.085936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8b2ed-e7d9-4687-9740-f008c8f9ca02 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:28.095032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e0d9f-5700-4dec-a735-d48040a26d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:28.099797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e0d9f-5700-4dec-a735-d48040a26d87 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:28.100086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e0d9f-5700-4dec-a735-d48040a26d87 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:28.100363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e0d9f-5700-4dec-a735-d48040a26d87 None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/c2f11dcf2c3c41a0b435a90af74c2e5a/roles/844d711adf9b4c1f8471a01e6031e1e6` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:28.122291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:28.133312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e0d9f-5700-4dec-a735-d48040a26d87 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=c2f11dcf2c3c41a0b435a90af74c2e5a, role_id=844d711adf9b4c1f8471a01e6031e1e6)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:28.139353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e0d9f-5700-4dec-a735-d48040a26d87 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:28.226333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8b2ed-e7d9-4687-9740-f008c8f9ca02 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:28.254111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 314/605] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:32:27 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/c2f11dcf2c3c41a0b435a90af74c2e5a/roles/844d711adf9b4c1f8471a01e6031e1e6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:32:28.275420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a8713-abb5-417b-bd95-b4b35a05d6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:28.276116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8713-abb5-417b-bd95-b4b35a05d6eb None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:28.276396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8713-abb5-417b-bd95-b4b35a05d6eb None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:28.276664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8713-abb5-417b-bd95-b4b35a05d6eb None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:28.277346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a8713-abb5-417b-bd95-b4b35a05d6eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:28.277643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a8713-abb5-417b-bd95-b4b35a05d6eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:28.279162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a8713-abb5-417b-bd95-b4b35a05d6eb None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:28.293865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 292/606] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:28.299986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e0d9f-5700-4dec-a735-d48040a26d87 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:28.324404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e050c-ba06-4a8b-8a3c-e461038cb96b None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:28.324689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e050c-ba06-4a8b-8a3c-e461038cb96b None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:28.325685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e050c-ba06-4a8b-8a3c-e461038cb96b None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:28.326536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 293/607] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:28.333882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bc8b2-b8e2-4628-86db-ce70c4626541 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:28.334348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bc8b2-b8e2-4628-86db-ce70c4626541 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:28.334780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975bc8b2-b8e2-4628-86db-ce70c4626541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:28.357852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-975bc8b2-b8e2-4628-86db-ce70c4626541 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:28.359806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-975bc8b2-b8e2-4628-86db-ce70c4626541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:28.443412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:28.445133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 294/608] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:32:28.482388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bc8b2-b8e2-4628-86db-ce70c4626541 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:28.535228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e0d9f-5700-4dec-a735-d48040a26d87 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:28.571867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44927901-339c-455b-98c8-70779e55aa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:28.572512 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44927901-339c-455b-98c8-70779e55aa97 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:28.572785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44927901-339c-455b-98c8-70779e55aa97 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:28.573051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44927901-339c-455b-98c8-70779e55aa97 None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/289e7caceab944e9b90f2e9298fb2cf1/roles/844d711adf9b4c1f8471a01e6031e1e6` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:28.594709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44927901-339c-455b-98c8-70779e55aa97 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=289e7caceab944e9b90f2e9298fb2cf1, role_id=844d711adf9b4c1f8471a01e6031e1e6)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:28.596065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44927901-339c-455b-98c8-70779e55aa97 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:28.598445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:28.618641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975bc8b2-b8e2-4628-86db-ce70c4626541 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:28.653494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a37927-b40a-4edf-9211-d7a5b7969e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:28.654150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37927-b40a-4edf-9211-d7a5b7969e1f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:28.654400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37927-b40a-4edf-9211-d7a5b7969e1f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:28.654646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a37927-b40a-4edf-9211-d7a5b7969e1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:28.665796 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a37927-b40a-4edf-9211-d7a5b7969e1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-c3a37927-b40a-4edf-9211-d7a5b7969e1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:28.670380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:28.671336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:28.672198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 295/609] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:28.678258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad9f60-ccfb-4ea6-9df9-358607f8341f None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:28.678258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad9f60-ccfb-4ea6-9df9-358607f8341f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:28.678258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad9f60-ccfb-4ea6-9df9-358607f8341f None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:28.679154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 296/610] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:28.679998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad9f60-ccfb-4ea6-9df9-358607f8341f None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:28.708535 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 315/611] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:32:28 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/289e7caceab944e9b90f2e9298fb2cf1/roles/844d711adf9b4c1f8471a01e6031e1e6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:28.718977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bef86-8010-43ff-977e-0933ae045918 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:28.719344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bef86-8010-43ff-977e-0933ae045918 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:28.719616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bef86-8010-43ff-977e-0933ae045918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:28.760217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ab5bef86-8010-43ff-977e-0933ae045918 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:28.764709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ab5bef86-8010-43ff-977e-0933ae045918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:28.795405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:28.889938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad9f60-ccfb-4ea6-9df9-358607f8341f None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:28.895869 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 316/612] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:28.927929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bef86-8010-43ff-977e-0933ae045918 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:28.935119 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4509638-a76c-4142-803b-7d9e816cf067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:28.936054 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4509638-a76c-4142-803b-7d9e816cf067 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:28.936525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4509638-a76c-4142-803b-7d9e816cf067 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:28.937520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4509638-a76c-4142-803b-7d9e816cf067 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:28.938459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4509638-a76c-4142-803b-7d9e816cf067 None admin] RBAC: Adding query filter params (name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:28.938975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4509638-a76c-4142-803b-7d9e816cf067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:28.941263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4509638-a76c-4142-803b-7d9e816cf067 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:28.953058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 297/613] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:28.958822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4509638-a76c-4142-803b-7d9e816cf067 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:29.062764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bef86-8010-43ff-977e-0933ae045918 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:29.092366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:29.105158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6c5e1-573e-4c71-b51e-03691c5f740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:29.106202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6c5e1-573e-4c71-b51e-03691c5f740a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.106673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6c5e1-573e-4c71-b51e-03691c5f740a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.107413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6c5e1-573e-4c71-b51e-03691c5f740a None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.119887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6c5e1-573e-4c71-b51e-03691c5f740a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-0ea6c5e1-573e-4c71-b51e-03691c5f740a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:29.125990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:29.126907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:29.128154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:29.132523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 317/614] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:29.139835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4509638-a76c-4142-803b-7d9e816cf067 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:29.145729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6c5e1-573e-4c71-b51e-03691c5f740a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:29.173140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:29.198524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0450500-1a71-4f2c-919b-0cf77803c91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:29.199151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450500-1a71-4f2c-919b-0cf77803c91c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.199424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450500-1a71-4f2c-919b-0cf77803c91c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.199688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0450500-1a71-4f2c-919b-0cf77803c91c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.222159 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0450500-1a71-4f2c-919b-0cf77803c91c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:29.223595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0450500-1a71-4f2c-919b-0cf77803c91c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-a0450500-1a71-4f2c-919b-0cf77803c91c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:29.238390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:29.239343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:29.240333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:29.240333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:29.240333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:29.240333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 298/615] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:29.242004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8970930-43bc-438f-a35e-c60f12b44b04 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.242004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8970930-43bc-438f-a35e-c60f12b44b04 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.242308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8970930-43bc-438f-a35e-c60f12b44b04 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.243944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 299/616] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:29.249865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789bd2d-9f90-4ae5-8ca7-ae032ecd8c01 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.250235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789bd2d-9f90-4ae5-8ca7-ae032ecd8c01 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.250572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789bd2d-9f90-4ae5-8ca7-ae032ecd8c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.273284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-a789bd2d-9f90-4ae5-8ca7-ae032ecd8c01 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:29.275218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a789bd2d-9f90-4ae5-8ca7-ae032ecd8c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:29.295775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6c5e1-573e-4c71-b51e-03691c5f740a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:29.347263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1f790-5899-4ebc-bf8c-3c405ac28ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:29.348212 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f790-5899-4ebc-bf8c-3c405ac28ac7 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.348560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f790-5899-4ebc-bf8c-3c405ac28ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.349081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f790-5899-4ebc-bf8c-3c405ac28ac7 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.350192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1f790-5899-4ebc-bf8c-3c405ac28ac7 None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:29.350800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1f790-5899-4ebc-bf8c-3c405ac28ac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:29.352453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1f790-5899-4ebc-bf8c-3c405ac28ac7 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:29.357326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 300/617] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 09 16:32:29.358191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789bd2d-9f90-4ae5-8ca7-ae032ecd8c01 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:29.371860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 318/618] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:29.378782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1f790-5899-4ebc-bf8c-3c405ac28ac7 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:29.442348 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:29.502201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1f790-5899-4ebc-bf8c-3c405ac28ac7 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:29.510591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789bd2d-9f90-4ae5-8ca7-ae032ecd8c01 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:29.512989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:29.542516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709fefa-8378-43fc-bcdb-b319354531a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:29.543267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709fefa-8378-43fc-bcdb-b319354531a5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.543649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709fefa-8378-43fc-bcdb-b319354531a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.544366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709fefa-8378-43fc-bcdb-b319354531a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.556900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709fefa-8378-43fc-bcdb-b319354531a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:29.558403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8709fefa-8378-43fc-bcdb-b319354531a5 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:29.562923 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528d5aa-5b69-416c-a257-30530ee6cc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:29.563709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528d5aa-5b69-416c-a257-30530ee6cc6f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.564156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528d5aa-5b69-416c-a257-30530ee6cc6f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.564627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528d5aa-5b69-416c-a257-30530ee6cc6f None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.565431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528d5aa-5b69-416c-a257-30530ee6cc6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:29.566422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528d5aa-5b69-416c-a257-30530ee6cc6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:29.568534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528d5aa-5b69-416c-a257-30530ee6cc6f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-8709fefa-8378-43fc-bcdb-b319354531a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:29.574734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:29.575987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:29.577151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:29.577151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:29.577151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:29.583081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 319/619] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:29.583081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709fefa-8378-43fc-bcdb-b319354531a5 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:29.592305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 301/620] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:29.593669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528d5aa-5b69-416c-a257-30530ee6cc6f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:29.714329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:29.721080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709fefa-8378-43fc-bcdb-b319354531a5 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:29.727734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528d5aa-5b69-416c-a257-30530ee6cc6f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:29.765209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001fb8a-9ff3-4ed2-a9c8-a9d29061589f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:29.765815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001fb8a-9ff3-4ed2-a9c8-a9d29061589f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.766126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001fb8a-9ff3-4ed2-a9c8-a9d29061589f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.766395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001fb8a-9ff3-4ed2-a9c8-a9d29061589f None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.767070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983697b8-810e-4672-b3e7-c4905ab19c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:29.767209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001fb8a-9ff3-4ed2-a9c8-a9d29061589f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:29.767386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001fb8a-9ff3-4ed2-a9c8-a9d29061589f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:29.768180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983697b8-810e-4672-b3e7-c4905ab19c67 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.768656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983697b8-810e-4672-b3e7-c4905ab19c67 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.768906 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001fb8a-9ff3-4ed2-a9c8-a9d29061589f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:29.769337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983697b8-810e-4672-b3e7-c4905ab19c67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.782840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983697b8-810e-4672-b3e7-c4905ab19c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:29.788683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 302/621] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:29.788683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001fb8a-9ff3-4ed2-a9c8-a9d29061589f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-983697b8-810e-4672-b3e7-c4905ab19c67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:29.790593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:29.791508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:29.792320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:29.796537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 320/622] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:29.797432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983697b8-810e-4672-b3e7-c4905ab19c67 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:29.818326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:29.898694 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001fb8a-9ff3-4ed2-a9c8-a9d29061589f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:29.905346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983697b8-810e-4672-b3e7-c4905ab19c67 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:29.935863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e246c50-a8b4-4cae-acdc-083627d02a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:29.936883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e246c50-a8b4-4cae-acdc-083627d02a64 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.937362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e246c50-a8b4-4cae-acdc-083627d02a64 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.937828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e246c50-a8b4-4cae-acdc-083627d02a64 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.943897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84303118-9082-476f-b9b1-ae098daa0361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:29.945127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84303118-9082-476f-b9b1-ae098daa0361 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:29.945481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84303118-9082-476f-b9b1-ae098daa0361 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:29.945717 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84303118-9082-476f-b9b1-ae098daa0361 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:29.947132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e246c50-a8b4-4cae-acdc-083627d02a64 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:29.948766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e246c50-a8b4-4cae-acdc-083627d02a64 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:29.954256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84303118-9082-476f-b9b1-ae098daa0361 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:29.955683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84303118-9082-476f-b9b1-ae098daa0361 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-0e246c50-a8b4-4cae-acdc-083627d02a64 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:29.961386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:29.962496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:29.963435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:29.963435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:29.963435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:29.963435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:29.963435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:29.963435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:29.963435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:29.963435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:29.963435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:29.963435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:29.965661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 303/623] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Aug 9 16:32:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:29.967326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e246c50-a8b4-4cae-acdc-083627d02a64 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-84303118-9082-476f-b9b1-ae098daa0361 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:29.967840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:29.968775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:29.969703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:29.969703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:29.969703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:29.969703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:29.969703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:29.969703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:29.969703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:29.969703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:29.969703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:29.969703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:29.974098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 321/624] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:29.975138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84303118-9082-476f-b9b1-ae098daa0361 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:30.003392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:30.064993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:30.084551 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e246c50-a8b4-4cae-acdc-083627d02a64 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:30.100728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84303118-9082-476f-b9b1-ae098daa0361 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:30.121253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5e6d8-840b-46f7-915c-eb55aa9c671e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:30.122055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5e6d8-840b-46f7-915c-eb55aa9c671e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.122482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5e6d8-840b-46f7-915c-eb55aa9c671e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.123081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5e6d8-840b-46f7-915c-eb55aa9c671e None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.123881 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5e6d8-840b-46f7-915c-eb55aa9c671e None admin] RBAC: Adding query filter params (name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:30.124294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5e6d8-840b-46f7-915c-eb55aa9c671e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:30.125764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5e6d8-840b-46f7-915c-eb55aa9c671e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:30.134897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 304/625] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:30.135837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5e6d8-840b-46f7-915c-eb55aa9c671e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:30.141501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7aea55-7baa-4115-b9a2-3f87489543fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:30.142329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7aea55-7baa-4115-b9a2-3f87489543fd None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.142799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7aea55-7baa-4115-b9a2-3f87489543fd None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.143236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7aea55-7baa-4115-b9a2-3f87489543fd None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.144104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7aea55-7baa-4115-b9a2-3f87489543fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:30.144590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7aea55-7baa-4115-b9a2-3f87489543fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:30.146292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7aea55-7baa-4115-b9a2-3f87489543fd None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:30.166328 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 322/626] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Aug 9 16:32:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:30.167474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7aea55-7baa-4115-b9a2-3f87489543fd None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:30.246448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5e6d8-840b-46f7-915c-eb55aa9c671e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:30.256030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:30.282443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808dd7d-0a9e-4ac1-83e2-8a4423cd62bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:30.283209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808dd7d-0a9e-4ac1-83e2-8a4423cd62bb None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.283593 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808dd7d-0a9e-4ac1-83e2-8a4423cd62bb None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.283967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808dd7d-0a9e-4ac1-83e2-8a4423cd62bb None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.284149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7aea55-7baa-4115-b9a2-3f87489543fd None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:30.284906 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808dd7d-0a9e-4ac1-83e2-8a4423cd62bb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:30.285310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808dd7d-0a9e-4ac1-83e2-8a4423cd62bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:30.287017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7808dd7d-0a9e-4ac1-83e2-8a4423cd62bb None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:30.310216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 305/627] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Aug 9 16:32:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:30.310216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808dd7d-0a9e-4ac1-83e2-8a4423cd62bb None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:30.315132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:30.331193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eec979-1de4-4d3a-926e-db35cd98b42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:30.332149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec979-1de4-4d3a-926e-db35cd98b42d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.332803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec979-1de4-4d3a-926e-db35cd98b42d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.333285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eec979-1de4-4d3a-926e-db35cd98b42d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.345417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eec979-1de4-4d3a-926e-db35cd98b42d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:30.347222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eec979-1de4-4d3a-926e-db35cd98b42d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-07eec979-1de4-4d3a-926e-db35cd98b42d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:30.363782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:30.365287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:30.366781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:30.366781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:30.366781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:30.369860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 323/628] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:30.371211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eec979-1de4-4d3a-926e-db35cd98b42d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:30.416189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808dd7d-0a9e-4ac1-83e2-8a4423cd62bb None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:30.453368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5de3a0-3ca1-42a6-95fa-4ff1f6b649b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:30.454415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5de3a0-3ca1-42a6-95fa-4ff1f6b649b6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.454802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5de3a0-3ca1-42a6-95fa-4ff1f6b649b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.455075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5de3a0-3ca1-42a6-95fa-4ff1f6b649b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.489538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eec979-1de4-4d3a-926e-db35cd98b42d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:30.522258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:30.528260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5de3a0-3ca1-42a6-95fa-4ff1f6b649b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:30.529807 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5de3a0-3ca1-42a6-95fa-4ff1f6b649b6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:30.531985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:30.533940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13572279-8239-4ccd-9122-1a62f6de6d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:30.534608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13572279-8239-4ccd-9122-1a62f6de6d75 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.534891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13572279-8239-4ccd-9122-1a62f6de6d75 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.535154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13572279-8239-4ccd-9122-1a62f6de6d75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.546252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13572279-8239-4ccd-9122-1a62f6de6d75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:30.547687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13572279-8239-4ccd-9122-1a62f6de6d75 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-13572279-8239-4ccd-9122-1a62f6de6d75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:30.563883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:30.564870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:30.566994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:30.566994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:30.566994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:30.566994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 324/629] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:30.566994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13572279-8239-4ccd-9122-1a62f6de6d75 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:30.647628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 306/630] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:30.648834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82f3f8-fe28-4d4c-8b27-0c079d79f9d8 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.649122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82f3f8-fe28-4d4c-8b27-0c079d79f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.649394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82f3f8-fe28-4d4c-8b27-0c079d79f9d8 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.650667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 307/631] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:30.651360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82f3f8-fe28-4d4c-8b27-0c079d79f9d8 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:30.678107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13572279-8239-4ccd-9122-1a62f6de6d75 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:30.714584 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e322b35-7345-44ef-8b7e-9dc01d57d6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:30.715348 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e322b35-7345-44ef-8b7e-9dc01d57d6aa None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.715733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e322b35-7345-44ef-8b7e-9dc01d57d6aa None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.716530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e322b35-7345-44ef-8b7e-9dc01d57d6aa None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.717241 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e322b35-7345-44ef-8b7e-9dc01d57d6aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:30.717648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e322b35-7345-44ef-8b7e-9dc01d57d6aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:30.719158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e322b35-7345-44ef-8b7e-9dc01d57d6aa None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:30.731459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 325/632] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:30.732387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e322b35-7345-44ef-8b7e-9dc01d57d6aa None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:30.763154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82f3f8-fe28-4d4c-8b27-0c079d79f9d8 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:30.765391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:30.778345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:30.802376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ea887-1e6c-4b2b-aa18-372f79b7ede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:30.803265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ea887-1e6c-4b2b-aa18-372f79b7ede6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.803889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ea887-1e6c-4b2b-aa18-372f79b7ede6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.804166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ea887-1e6c-4b2b-aa18-372f79b7ede6 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.804802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ea887-1e6c-4b2b-aa18-372f79b7ede6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:30.805104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ea887-1e6c-4b2b-aa18-372f79b7ede6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:30.806496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ea887-1e6c-4b2b-aa18-372f79b7ede6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:30.822849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 308/633] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:30.822849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf8bb1-b7f8-4a8c-9654-1b405cdf36a1 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.822849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf8bb1-b7f8-4a8c-9654-1b405cdf36a1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.822849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bf8bb1-b7f8-4a8c-9654-1b405cdf36a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.838494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d2bf8bb1-b7f8-4a8c-9654-1b405cdf36a1 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:30.839685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d2bf8bb1-b7f8-4a8c-9654-1b405cdf36a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:30.842712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e322b35-7345-44ef-8b7e-9dc01d57d6aa None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:30.878790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6a14f-5ad9-4fa7-94c9-d079e52c610c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:30.879708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6a14f-5ad9-4fa7-94c9-d079e52c610c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:30.880164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6a14f-5ad9-4fa7-94c9-d079e52c610c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:30.880641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6a14f-5ad9-4fa7-94c9-d079e52c610c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:30.891947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6a14f-5ad9-4fa7-94c9-d079e52c610c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:30.893614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6a14f-5ad9-4fa7-94c9-d079e52c610c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:30.908249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 309/634] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:30.909175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf8bb1-b7f8-4a8c-9654-1b405cdf36a1 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-3bf6a14f-5ad9-4fa7-94c9-d079e52c610c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:30.910235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:30.911206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:30.912146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:30.912146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:30.912146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:30.912146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 326/635] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:30.912366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6a14f-5ad9-4fa7-94c9-d079e52c610c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.013491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:31.019933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:31.024986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bf8bb1-b7f8-4a8c-9654-1b405cdf36a1 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.029043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6a14f-5ad9-4fa7-94c9-d079e52c610c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.060763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80efdc-b76c-44bb-92ad-f27bb5586f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.061021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc9dbe-4ea7-4397-ad8f-1a3959b1c6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.061372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80efdc-b76c-44bb-92ad-f27bb5586f48 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.061628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc9dbe-4ea7-4397-ad8f-1a3959b1c6c9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.061721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80efdc-b76c-44bb-92ad-f27bb5586f48 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.061897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80efdc-b76c-44bb-92ad-f27bb5586f48 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.062007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc9dbe-4ea7-4397-ad8f-1a3959b1c6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.062186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc9dbe-4ea7-4397-ad8f-1a3959b1c6c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.068628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc9dbe-4ea7-4397-ad8f-1a3959b1c6c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:31.068847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80efdc-b76c-44bb-92ad-f27bb5586f48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:31.070808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80efdc-b76c-44bb-92ad-f27bb5586f48 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:31.070993 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc9dbe-4ea7-4397-ad8f-1a3959b1c6c9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-0b80efdc-b76c-44bb-92ad-f27bb5586f48 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:31.081510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:31.082711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 310/636] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:31.083844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80efdc-b76c-44bb-92ad-f27bb5586f48 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-8bdc9dbe-4ea7-4397-ad8f-1a3959b1c6c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:31.085639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:31.086528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:31.089557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:31.089557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:31.089557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:31.089557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:31.089557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:31.089557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:31.089557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:31.089557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:31.089557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 327/637] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:31.089557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc9dbe-4ea7-4397-ad8f-1a3959b1c6c9 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.185254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80efdc-b76c-44bb-92ad-f27bb5586f48 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.188317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc9dbe-4ea7-4397-ad8f-1a3959b1c6c9 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.218804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d025d24-7ecf-4282-87f4-d63ce84c4ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.219389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d025d24-7ecf-4282-87f4-d63ce84c4ef7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.219664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d025d24-7ecf-4282-87f4-d63ce84c4ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.219929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d025d24-7ecf-4282-87f4-d63ce84c4ef7 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.220552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d025d24-7ecf-4282-87f4-d63ce84c4ef7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:31.220846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d025d24-7ecf-4282-87f4-d63ce84c4ef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:31.221628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d63211-801f-420e-bdf9-1f23034594a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.222209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d025d24-7ecf-4282-87f4-d63ce84c4ef7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:31.222304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d63211-801f-420e-bdf9-1f23034594a7 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.222528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d63211-801f-420e-bdf9-1f23034594a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.222790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d63211-801f-420e-bdf9-1f23034594a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.232570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d63211-801f-420e-bdf9-1f23034594a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:31.233411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 311/638] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:31.234586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ce9d1-494a-4b29-99b9-3e296ee5cb12 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.234863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ce9d1-494a-4b29-99b9-3e296ee5cb12 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.235129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ce9d1-494a-4b29-99b9-3e296ee5cb12 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.238073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 312/639] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:31.238073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ce9d1-494a-4b29-99b9-3e296ee5cb12 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-65d63211-801f-420e-bdf9-1f23034594a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:31.238284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:31.239257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 328/640] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:31.240097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d63211-801f-420e-bdf9-1f23034594a7 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.254841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:31.258984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:31.343413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ce9d1-494a-4b29-99b9-3e296ee5cb12 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.343880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d63211-801f-420e-bdf9-1f23034594a7 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.375612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c14131-a799-4eae-aa67-e8c169a7d634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.376234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c14131-a799-4eae-aa67-e8c169a7d634 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.376457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c14131-a799-4eae-aa67-e8c169a7d634 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.376725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c14131-a799-4eae-aa67-e8c169a7d634 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.376932 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c055ad-083a-47ca-88bc-55e700953bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.377402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c14131-a799-4eae-aa67-e8c169a7d634 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:31.377520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c055ad-083a-47ca-88bc-55e700953bb6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.377702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c14131-a799-4eae-aa67-e8c169a7d634 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:31.377799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c055ad-083a-47ca-88bc-55e700953bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.378072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c055ad-083a-47ca-88bc-55e700953bb6 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.378780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c055ad-083a-47ca-88bc-55e700953bb6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:31.379073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c055ad-083a-47ca-88bc-55e700953bb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:31.379160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c14131-a799-4eae-aa67-e8c169a7d634 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:31.380452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c055ad-083a-47ca-88bc-55e700953bb6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:31.391476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 329/641] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:31.392350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c14131-a799-4eae-aa67-e8c169a7d634 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.405095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 313/642] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Aug 9 16:32:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:31.405095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4351d5-5120-45d8-97cf-91b54437bf96 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.405095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4351d5-5120-45d8-97cf-91b54437bf96 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.405095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4351d5-5120-45d8-97cf-91b54437bf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.420048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-bf4351d5-5120-45d8-97cf-91b54437bf96 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:31.421246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bf4351d5-5120-45d8-97cf-91b54437bf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:31.466674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:31.482936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:31.487133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 314/643] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:31.487992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4351d5-5120-45d8-97cf-91b54437bf96 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.495784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c14131-a799-4eae-aa67-e8c169a7d634 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.539022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94134fc7-e621-4e67-9d6b-40705272ef09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.539635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94134fc7-e621-4e67-9d6b-40705272ef09 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.539908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94134fc7-e621-4e67-9d6b-40705272ef09 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.540177 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94134fc7-e621-4e67-9d6b-40705272ef09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.547519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94134fc7-e621-4e67-9d6b-40705272ef09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:31.548974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94134fc7-e621-4e67-9d6b-40705272ef09 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-94134fc7-e621-4e67-9d6b-40705272ef09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:31.560337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:31.561382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:31.562636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:31.562636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:31.562636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:31.562636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:31.562636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:31.562636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:31.562636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:31.562636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:31.566440 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 330/644] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:31.567234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94134fc7-e621-4e67-9d6b-40705272ef09 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.589190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4351d5-5120-45d8-97cf-91b54437bf96 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.622596 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14885f-de7d-45bf-8af8-43d4e2a25ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.623205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14885f-de7d-45bf-8af8-43d4e2a25ddd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.623461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14885f-de7d-45bf-8af8-43d4e2a25ddd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.623726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14885f-de7d-45bf-8af8-43d4e2a25ddd None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.624380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14885f-de7d-45bf-8af8-43d4e2a25ddd None admin] RBAC: Adding query filter params (name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:31.624680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14885f-de7d-45bf-8af8-43d4e2a25ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:31.626050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14885f-de7d-45bf-8af8-43d4e2a25ddd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:31.634293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 315/645] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:31.635058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14885f-de7d-45bf-8af8-43d4e2a25ddd None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.664896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94134fc7-e621-4e67-9d6b-40705272ef09 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.692872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:31.696978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:31.698255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ca787-7e36-42a5-99a9-d4ce40c7dadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.715363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ca787-7e36-42a5-99a9-d4ce40c7dadd None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.715363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ca787-7e36-42a5-99a9-d4ce40c7dadd None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.715363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ca787-7e36-42a5-99a9-d4ce40c7dadd None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/35371b488c474b7f851af16bdad34f69/roles/844d711adf9b4c1f8471a01e6031e1e6` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.715363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ca787-7e36-42a5-99a9-d4ce40c7dadd None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=35371b488c474b7f851af16bdad34f69, role_id=844d711adf9b4c1f8471a01e6031e1e6)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:31.715363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ca787-7e36-42a5-99a9-d4ce40c7dadd None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:31.735235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14885f-de7d-45bf-8af8-43d4e2a25ddd None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.767549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372a1e4-b658-42c5-8435-c2e8fe682ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.768149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372a1e4-b658-42c5-8435-c2e8fe682ded None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.768419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372a1e4-b658-42c5-8435-c2e8fe682ded None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.768684 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372a1e4-b658-42c5-8435-c2e8fe682ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.785209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 331/646] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:32:31 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/35371b488c474b7f851af16bdad34f69/roles/844d711adf9b4c1f8471a01e6031e1e6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:31.786162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ca787-7e36-42a5-99a9-d4ce40c7dadd None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.848170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372a1e4-b658-42c5-8435-c2e8fe682ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:31.849626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372a1e4-b658-42c5-8435-c2e8fe682ded None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:31.915573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:31.916961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ca787-7e36-42a5-99a9-d4ce40c7dadd None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:31.961496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf916fa2-5467-4ae2-9cf3-bbad9cc2bc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:31.962184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf916fa2-5467-4ae2-9cf3-bbad9cc2bc5e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:31.962466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf916fa2-5467-4ae2-9cf3-bbad9cc2bc5e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:31.962733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf916fa2-5467-4ae2-9cf3-bbad9cc2bc5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:31.966436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 316/647] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:31.966436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372a1e4-b658-42c5-8435-c2e8fe682ded None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:31.976317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf916fa2-5467-4ae2-9cf3-bbad9cc2bc5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-cf916fa2-5467-4ae2-9cf3-bbad9cc2bc5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:31.981052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:31.982195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:31.983053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 332/648] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:31.983760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf916fa2-5467-4ae2-9cf3-bbad9cc2bc5e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.034849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:32.078554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372a1e4-b658-42c5-8435-c2e8fe682ded None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:32.093657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf916fa2-5467-4ae2-9cf3-bbad9cc2bc5e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:32.115080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdde89-18e0-45de-ac71-891d11accf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:32.115675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdde89-18e0-45de-ac71-891d11accf9f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:32.115948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdde89-18e0-45de-ac71-891d11accf9f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:32.116237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fdde89-18e0-45de-ac71-891d11accf9f None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:32.117452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdde89-18e0-45de-ac71-891d11accf9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:32.117747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdde89-18e0-45de-ac71-891d11accf9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:32.119184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fdde89-18e0-45de-ac71-891d11accf9f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:32.135715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 317/649] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:32.135715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdde89-18e0-45de-ac71-891d11accf9f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.138955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd330f9-5248-4dc4-ba2a-ddbf8d2f06e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:32.139406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd330f9-5248-4dc4-ba2a-ddbf8d2f06e3 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:32.139770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd330f9-5248-4dc4-ba2a-ddbf8d2f06e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:32.140948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd330f9-5248-4dc4-ba2a-ddbf8d2f06e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:32.151285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd330f9-5248-4dc4-ba2a-ddbf8d2f06e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:32.152828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd330f9-5248-4dc4-ba2a-ddbf8d2f06e3 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:32.164566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-acd330f9-5248-4dc4-ba2a-ddbf8d2f06e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:32.167928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:32.169041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:32.170155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:32.170155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:32.170155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:32.179479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 333/650] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:32.179479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd330f9-5248-4dc4-ba2a-ddbf8d2f06e3 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.253186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fdde89-18e0-45de-ac71-891d11accf9f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:32.273874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:32.280981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd330f9-5248-4dc4-ba2a-ddbf8d2f06e3 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:32.295778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e9c1-d503-468b-ab37-a196842f30a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:32.296867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e9c1-d503-468b-ab37-a196842f30a4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:32.297548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e9c1-d503-468b-ab37-a196842f30a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:32.298002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e9c1-d503-468b-ab37-a196842f30a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:32.314173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8e9c1-d503-468b-ab37-a196842f30a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:32.316293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8e9c1-d503-468b-ab37-a196842f30a4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:32.316456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848e59b-a5f1-46d1-8667-c4fb6e277056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:32.317014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848e59b-a5f1-46d1-8667-c4fb6e277056 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:32.317301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848e59b-a5f1-46d1-8667-c4fb6e277056 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:32.317576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a848e59b-a5f1-46d1-8667-c4fb6e277056 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:32.318311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848e59b-a5f1-46d1-8667-c4fb6e277056 None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:32.318696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848e59b-a5f1-46d1-8667-c4fb6e277056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:32.320153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a848e59b-a5f1-46d1-8667-c4fb6e277056 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:32.328702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 334/651] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:32.329594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848e59b-a5f1-46d1-8667-c4fb6e277056 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-bfb8e9c1-d503-468b-ab37-a196842f30a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:32.343305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:32.345158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:32.346269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:32.346269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:32.346269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:32.346269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 318/652] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:32.346269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e9c1-d503-468b-ab37-a196842f30a4 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.430017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:32.453049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a848e59b-a5f1-46d1-8667-c4fb6e277056 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:32.453643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e9c1-d503-468b-ab37-a196842f30a4 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:32.488459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b5b9a-eb96-4bcc-8ad7-a2f0aac9cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:32.489058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b5b9a-eb96-4bcc-8ad7-a2f0aac9cef5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:32.489334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b5b9a-eb96-4bcc-8ad7-a2f0aac9cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:32.489591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b5b9a-eb96-4bcc-8ad7-a2f0aac9cef5 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:32.490182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd559a-3769-48e7-8ea4-d624951a173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:32.490321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b5b9a-eb96-4bcc-8ad7-a2f0aac9cef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:32.490653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b5b9a-eb96-4bcc-8ad7-a2f0aac9cef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:32.491034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd559a-3769-48e7-8ea4-d624951a173f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:32.491433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd559a-3769-48e7-8ea4-d624951a173f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:32.491787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd559a-3769-48e7-8ea4-d624951a173f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:32.492009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b5b9a-eb96-4bcc-8ad7-a2f0aac9cef5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:32.492679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd559a-3769-48e7-8ea4-d624951a173f None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=5d17aa4b23e04290b6c8b1924b0dcf32) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:32.493097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd559a-3769-48e7-8ea4-d624951a173f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:32.494613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd559a-3769-48e7-8ea4-d624951a173f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:32.506176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 335/653] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:32 2022] GET /identity/v3/role_assignments?user.id=5d17aa4b23e04290b6c8b1924b0dcf32&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:32.506176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd559a-3769-48e7-8ea4-d624951a173f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.522065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 319/654] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:32.523712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b5b9a-eb96-4bcc-8ad7-a2f0aac9cef5 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.542457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:32.642145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b5b9a-eb96-4bcc-8ad7-a2f0aac9cef5 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:32.648565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd559a-3769-48e7-8ea4-d624951a173f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:32.683823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd3f01-6b22-4aee-9b41-f016def72e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:32.684344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd3f01-6b22-4aee-9b41-f016def72e11 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:32.684628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd3f01-6b22-4aee-9b41-f016def72e11 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:32.684898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd3f01-6b22-4aee-9b41-f016def72e11 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:32.686714 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd3f01-6b22-4aee-9b41-f016def72e11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:32.686714 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd3f01-6b22-4aee-9b41-f016def72e11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:32.688684 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd3f01-6b22-4aee-9b41-f016def72e11 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:32.690419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:32.692743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e53bd-7ec9-486e-890d-26eeab4f0231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:32.693459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e53bd-7ec9-486e-890d-26eeab4f0231 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:32.693789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e53bd-7ec9-486e-890d-26eeab4f0231 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:32.694150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e53bd-7ec9-486e-890d-26eeab4f0231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:32.703548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 320/655] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:32.705192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd3f01-6b22-4aee-9b41-f016def72e11 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.707962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e53bd-7ec9-486e-890d-26eeab4f0231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:32.709521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e53bd-7ec9-486e-890d-26eeab4f0231 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-f90e53bd-7ec9-486e-890d-26eeab4f0231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:32.725985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:32.726994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:32.727872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:32.727872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:32.727872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:32.727872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 336/656] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:32.728539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e53bd-7ec9-486e-890d-26eeab4f0231 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.801200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:32.824875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd3f01-6b22-4aee-9b41-f016def72e11 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:32.839546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e53bd-7ec9-486e-890d-26eeab4f0231 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:32.864504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fad529-4eb5-4ed0-9ed7-e0f0620bc332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:32.865137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fad529-4eb5-4ed0-9ed7-e0f0620bc332 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:32.865406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fad529-4eb5-4ed0-9ed7-e0f0620bc332 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:32.865675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fad529-4eb5-4ed0-9ed7-e0f0620bc332 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:32.876533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fad529-4eb5-4ed0-9ed7-e0f0620bc332 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:32.878374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fad529-4eb5-4ed0-9ed7-e0f0620bc332 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:32.879718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92368a9-f445-43b5-82c4-bab0d7e26023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:32.880346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92368a9-f445-43b5-82c4-bab0d7e26023 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:32.880623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92368a9-f445-43b5-82c4-bab0d7e26023 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:32.880892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92368a9-f445-43b5-82c4-bab0d7e26023 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:32.888594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92368a9-f445-43b5-82c4-bab0d7e26023 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:32.890069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92368a9-f445-43b5-82c4-bab0d7e26023 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-b5fad529-4eb5-4ed0-9ed7-e0f0620bc332 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:32.892699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:32.893678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:32.894672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:32.894672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:32.894672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:32.894672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:32.894672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:32.894672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:32.894672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:32.894672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:32.894672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 09 16:32:32.894672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:32.897196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 321/657] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:32.898988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fad529-4eb5-4ed0-9ed7-e0f0620bc332 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-e92368a9-f445-43b5-82c4-bab0d7e26023 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:32.902682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:32.903568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:32.904926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:32.904926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:32.904926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:32.904926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:32.904926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:32.904926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:32.904926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:32.904926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:32.904926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 337/658] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:32.905430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92368a9-f445-43b5-82c4-bab0d7e26023 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:32.968076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:33.016900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fad529-4eb5-4ed0-9ed7-e0f0620bc332 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:33.053588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92368a9-f445-43b5-82c4-bab0d7e26023 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:33.055627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f171e-7ac1-4785-b634-08307762aa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:33.056582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f171e-7ac1-4785-b634-08307762aa9f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:33.057081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f171e-7ac1-4785-b634-08307762aa9f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:33.057568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f171e-7ac1-4785-b634-08307762aa9f None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:33.058466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f171e-7ac1-4785-b634-08307762aa9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:33.058970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f171e-7ac1-4785-b634-08307762aa9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:33.060907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f171e-7ac1-4785-b634-08307762aa9f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:33.074796 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 322/659] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:33.076195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f171e-7ac1-4785-b634-08307762aa9f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:33.086322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:33.096106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f1749-10ab-45db-b2d5-46cb52cf889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:33.096891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f1749-10ab-45db-b2d5-46cb52cf889c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:33.103072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f1749-10ab-45db-b2d5-46cb52cf889c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:33.103072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f1749-10ab-45db-b2d5-46cb52cf889c None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:33.103072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f1749-10ab-45db-b2d5-46cb52cf889c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:33.103072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f1749-10ab-45db-b2d5-46cb52cf889c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:33.103072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f1749-10ab-45db-b2d5-46cb52cf889c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:33.130106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 338/660] 10.222.0.79 () {56 vars in 1139 bytes} [Tue Aug 9 16:32:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:33.132155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f1749-10ab-45db-b2d5-46cb52cf889c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:33.193586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f171e-7ac1-4785-b634-08307762aa9f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:33.209395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:33.232804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a83dc3-4d71-4c18-b196-25ed012c4ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:33.233550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a83dc3-4d71-4c18-b196-25ed012c4ed7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:33.234153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a83dc3-4d71-4c18-b196-25ed012c4ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:33.234526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a83dc3-4d71-4c18-b196-25ed012c4ed7 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:33.235379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a83dc3-4d71-4c18-b196-25ed012c4ed7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:33.235778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a83dc3-4d71-4c18-b196-25ed012c4ed7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:33.237318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a83dc3-4d71-4c18-b196-25ed012c4ed7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:33.250442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 323/661] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:33.251466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a83dc3-4d71-4c18-b196-25ed012c4ed7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:33.267030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f1749-10ab-45db-b2d5-46cb52cf889c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:33.308427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a58ec-29d6-48d2-9529-c32466f4455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:33.309208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a58ec-29d6-48d2-9529-c32466f4455e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:33.309641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a58ec-29d6-48d2-9529-c32466f4455e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:33.310036 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a58ec-29d6-48d2-9529-c32466f4455e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:33.317650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a58ec-29d6-48d2-9529-c32466f4455e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:33.319267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50a58ec-29d6-48d2-9529-c32466f4455e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-a50a58ec-29d6-48d2-9529-c32466f4455e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:33.331842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:33.333602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 339/662] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:33.336368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a58ec-29d6-48d2-9529-c32466f4455e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:33.374781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a83dc3-4d71-4c18-b196-25ed012c4ed7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:33.388948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:33.422151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37db3c-3666-4adc-a63f-d229d523a0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:33.422788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37db3c-3666-4adc-a63f-d229d523a0c1 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:33.423966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37db3c-3666-4adc-a63f-d229d523a0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:33.424243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37db3c-3666-4adc-a63f-d229d523a0c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:33.448917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50a58ec-29d6-48d2-9529-c32466f4455e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:33.485843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:33.486154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5b9c8-f4fd-476f-9533-efacf811f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:33.486805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5b9c8-f4fd-476f-9533-efacf811f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:33.487106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5b9c8-f4fd-476f-9533-efacf811f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:33.487394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5b9c8-f4fd-476f-9533-efacf811f1d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:33.488231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5b9c8-f4fd-476f-9533-efacf811f1d8 None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=33003a9bf4454f5bac2e6ad73544efc4) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:33.488632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5b9c8-f4fd-476f-9533-efacf811f1d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:33.490130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5b9c8-f4fd-476f-9533-efacf811f1d8 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:33.497202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 340/663] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:33 2022] GET /identity/v3/role_assignments?user.id=33003a9bf4454f5bac2e6ad73544efc4&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:33.500288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5b9c8-f4fd-476f-9533-efacf811f1d8 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:33.534454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37db3c-3666-4adc-a63f-d229d523a0c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:33.536246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37db3c-3666-4adc-a63f-d229d523a0c1 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:33.632594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5b9c8-f4fd-476f-9533-efacf811f1d8 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:33.648376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:33.664933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 324/664] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:33.671757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37db3c-3666-4adc-a63f-d229d523a0c1 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:33.677983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abdfe5-be0e-4268-b3db-8eed6a4d8a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:33.677983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abdfe5-be0e-4268-b3db-8eed6a4d8a86 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:33.677983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abdfe5-be0e-4268-b3db-8eed6a4d8a86 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:33.677983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abdfe5-be0e-4268-b3db-8eed6a4d8a86 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:33.678413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abdfe5-be0e-4268-b3db-8eed6a4d8a86 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:33.678413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abdfe5-be0e-4268-b3db-8eed6a4d8a86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:33.678550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abdfe5-be0e-4268-b3db-8eed6a4d8a86 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:33.725043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 341/665] 10.222.0.79 () {56 vars in 1137 bytes} [Tue Aug 9 16:32:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:33.730857 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abdfe5-be0e-4268-b3db-8eed6a4d8a86 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:33.762102 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:33.789337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37db3c-3666-4adc-a63f-d229d523a0c1 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:33.829047 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455e022-8dc8-46d9-9aea-a70863c727d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:33.830511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e022-8dc8-46d9-9aea-a70863c727d5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:33.831018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e022-8dc8-46d9-9aea-a70863c727d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:33.831484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e022-8dc8-46d9-9aea-a70863c727d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:33.845229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455e022-8dc8-46d9-9aea-a70863c727d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:33.847149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455e022-8dc8-46d9-9aea-a70863c727d5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-c455e022-8dc8-46d9-9aea-a70863c727d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:33.862895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:33.863877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:33.864786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:33.864786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:33.864786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:33.868607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 325/666] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:33.868607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d6e08-a563-4838-b561-94bd6d36cee7 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:33.868607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d6e08-a563-4838-b561-94bd6d36cee7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:33.868607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d6e08-a563-4838-b561-94bd6d36cee7 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:33.870249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 326/667] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:33.872406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d6e08-a563-4838-b561-94bd6d36cee7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:33.889452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abdfe5-be0e-4268-b3db-8eed6a4d8a86 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:33.936407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5c18-5ec1-4c64-9809-ff698f00debb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:33.937030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5c18-5ec1-4c64-9809-ff698f00debb None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:33.937302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5c18-5ec1-4c64-9809-ff698f00debb None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:33.937567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db5c18-5ec1-4c64-9809-ff698f00debb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:33.990645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:34.004191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d6e08-a563-4838-b561-94bd6d36cee7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:34.019801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db5c18-5ec1-4c64-9809-ff698f00debb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:34.021165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db5c18-5ec1-4c64-9809-ff698f00debb None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:34.032103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:34.041976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c7213-5575-4b43-96be-94a97da2537f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:34.042680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c7213-5575-4b43-96be-94a97da2537f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.048347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c7213-5575-4b43-96be-94a97da2537f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.048347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c7213-5575-4b43-96be-94a97da2537f None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.048347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c7213-5575-4b43-96be-94a97da2537f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:34.048347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c7213-5575-4b43-96be-94a97da2537f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:34.048347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c7213-5575-4b43-96be-94a97da2537f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:34.059609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 327/668] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:34.061282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019167-0259-402d-be72-1940c42ad59d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.061741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019167-0259-402d-be72-1940c42ad59d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.062214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019167-0259-402d-be72-1940c42ad59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.082357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-45019167-0259-402d-be72-1940c42ad59d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:34.083994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-45019167-0259-402d-be72-1940c42ad59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:34.134015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 342/669] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 16:32:34.134015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5c18-5ec1-4c64-9809-ff698f00debb None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:34.169407 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 328/670] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:34.170546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45019167-0259-402d-be72-1940c42ad59d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:34.211262 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:34.243388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db5c18-5ec1-4c64-9809-ff698f00debb None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:34.279539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e5e6b-2216-4bc8-9b5b-0822d16f7770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:34.280271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5e6b-2216-4bc8-9b5b-0822d16f7770 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.280612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5e6b-2216-4bc8-9b5b-0822d16f7770 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.281052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e5e6b-2216-4bc8-9b5b-0822d16f7770 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.289000 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e5e6b-2216-4bc8-9b5b-0822d16f7770 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:34.290474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e5e6b-2216-4bc8-9b5b-0822d16f7770 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:34.291246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45019167-0259-402d-be72-1940c42ad59d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-4a5e5e6b-2216-4bc8-9b5b-0822d16f7770 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:34.302701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:34.303676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:34.305143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:34.305143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:34.305143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:34.305143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:34.305143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:34.305143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:34.305143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:34.305143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:34.309414 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 343/671] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:34.310413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e5e6b-2216-4bc8-9b5b-0822d16f7770 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:34.313770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:34.336292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e2bb8-b042-4f90-bd09-051a9bd8aa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:34.337074 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e2bb8-b042-4f90-bd09-051a9bd8aa84 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.339773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e2bb8-b042-4f90-bd09-051a9bd8aa84 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.339773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e2bb8-b042-4f90-bd09-051a9bd8aa84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.349260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e2bb8-b042-4f90-bd09-051a9bd8aa84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:34.351424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e2bb8-b042-4f90-bd09-051a9bd8aa84 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-3c9e2bb8-b042-4f90-bd09-051a9bd8aa84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:34.369235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:34.370265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:34.371260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:34.371260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:34.371260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:34.371260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 329/672] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:34.372500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e2bb8-b042-4f90-bd09-051a9bd8aa84 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:34.415822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e5e6b-2216-4bc8-9b5b-0822d16f7770 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:34.447338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:34.452251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee59ed-6ffa-4eea-8c43-a78d76d45c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:34.452862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee59ed-6ffa-4eea-8c43-a78d76d45c03 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.453134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee59ed-6ffa-4eea-8c43-a78d76d45c03 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.453399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee59ed-6ffa-4eea-8c43-a78d76d45c03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.464781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee59ed-6ffa-4eea-8c43-a78d76d45c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-fdee59ed-6ffa-4eea-8c43-a78d76d45c03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:34.470054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:34.471008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:34.472166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 344/673] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:34.472895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee59ed-6ffa-4eea-8c43-a78d76d45c03 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:34.488473 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e2bb8-b042-4f90-bd09-051a9bd8aa84 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:34.526479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e1e1a-88ec-4183-b297-59f68ab38670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:34.527504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e1e1a-88ec-4183-b297-59f68ab38670 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.528044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e1e1a-88ec-4183-b297-59f68ab38670 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.528517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e1e1a-88ec-4183-b297-59f68ab38670 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.529763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e1e1a-88ec-4183-b297-59f68ab38670 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:34.530284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e1e1a-88ec-4183-b297-59f68ab38670 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:34.531969 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0e1e1a-88ec-4183-b297-59f68ab38670 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:34.551781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 330/674] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:34.553030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e1e1a-88ec-4183-b297-59f68ab38670 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:34.583428 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:34.589376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee59ed-6ffa-4eea-8c43-a78d76d45c03 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:34.627528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3830a3-e778-4432-bbd8-bb2eac47c14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:34.628404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3830a3-e778-4432-bbd8-bb2eac47c14e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.628833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3830a3-e778-4432-bbd8-bb2eac47c14e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.629251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3830a3-e778-4432-bbd8-bb2eac47c14e None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.630052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3830a3-e778-4432-bbd8-bb2eac47c14e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:34.630597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3830a3-e778-4432-bbd8-bb2eac47c14e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:34.632228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3830a3-e778-4432-bbd8-bb2eac47c14e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:34.646637 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 345/675] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:34.648387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3830a3-e778-4432-bbd8-bb2eac47c14e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:34.664914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e1e1a-88ec-4183-b297-59f68ab38670 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:34.702103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c965c68-0cbd-4d84-8880-21bb376f14a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:34.702847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c965c68-0cbd-4d84-8880-21bb376f14a5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.703264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c965c68-0cbd-4d84-8880-21bb376f14a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.703624 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c965c68-0cbd-4d84-8880-21bb376f14a5 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.704428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:34.705621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c965c68-0cbd-4d84-8880-21bb376f14a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:34.706202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c965c68-0cbd-4d84-8880-21bb376f14a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:34.707838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c965c68-0cbd-4d84-8880-21bb376f14a5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:34.717803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 331/676] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:34.718900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c965c68-0cbd-4d84-8880-21bb376f14a5 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:34.754718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3830a3-e778-4432-bbd8-bb2eac47c14e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:34.790520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3491ba1-d4ad-4c92-bb81-1f720604f1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:34.791031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3491ba1-d4ad-4c92-bb81-1f720604f1ef None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.791302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3491ba1-d4ad-4c92-bb81-1f720604f1ef None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.791566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3491ba1-d4ad-4c92-bb81-1f720604f1ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.792732 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3491ba1-d4ad-4c92-bb81-1f720604f1ef None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=c2f11dcf2c3c41a0b435a90af74c2e5a) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:34.792732 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3491ba1-d4ad-4c92-bb81-1f720604f1ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:34.794089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3491ba1-d4ad-4c92-bb81-1f720604f1ef None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:34.806269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 346/677] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:34 2022] GET /identity/v3/role_assignments?user.id=c2f11dcf2c3c41a0b435a90af74c2e5a&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:34.807424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3491ba1-d4ad-4c92-bb81-1f720604f1ef None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:34.820658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:34.846847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c965c68-0cbd-4d84-8880-21bb376f14a5 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:34.911115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7d174-9483-438b-9222-61286dfff9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:34.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7d174-9483-438b-9222-61286dfff9a7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.912309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7d174-9483-438b-9222-61286dfff9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.912587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7d174-9483-438b-9222-61286dfff9a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.923144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7d174-9483-438b-9222-61286dfff9a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:34.924909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7d174-9483-438b-9222-61286dfff9a7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-bab7d174-9483-438b-9222-61286dfff9a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:34.941279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:34.942282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:34.943162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:34.943162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:34.943162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:34.943162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:34.943162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:34.943162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:34.943162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:34.943162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:34.943604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 332/678] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:34.945373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c0384-7004-4058-a71f-08f77bc3684c None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:34.945660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c0384-7004-4058-a71f-08f77bc3684c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:34.945933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c0384-7004-4058-a71f-08f77bc3684c None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:34.950043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 333/679] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:34.951149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c0384-7004-4058-a71f-08f77bc3684c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:34.970719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3491ba1-d4ad-4c92-bb81-1f720604f1ef None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:34.990367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:35.013106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1bd48-cfa5-44a1-b821-56182cba2e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:35.013839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1bd48-cfa5-44a1-b821-56182cba2e30 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:35.014192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1bd48-cfa5-44a1-b821-56182cba2e30 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:35.014499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1bd48-cfa5-44a1-b821-56182cba2e30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:35.027667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1bd48-cfa5-44a1-b821-56182cba2e30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:35.029079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1bd48-cfa5-44a1-b821-56182cba2e30 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-15c1bd48-cfa5-44a1-b821-56182cba2e30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:35.044433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:35.045481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:35.046793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:35.046793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:35.046793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:35.046793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 347/680] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:35.051299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414f553-b84b-45cd-8506-8ea576dbb7fe None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:35.051580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414f553-b84b-45cd-8506-8ea576dbb7fe None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:35.051850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414f553-b84b-45cd-8506-8ea576dbb7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:35.071226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d414f553-b84b-45cd-8506-8ea576dbb7fe None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:35.072381 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d414f553-b84b-45cd-8506-8ea576dbb7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:35.086529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c0384-7004-4058-a71f-08f77bc3684c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:35.126619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d8b3f-478c-44dc-9afc-b26239cb6278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:35.127257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d8b3f-478c-44dc-9afc-b26239cb6278 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:35.127531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d8b3f-478c-44dc-9afc-b26239cb6278 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:35.127797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d8b3f-478c-44dc-9afc-b26239cb6278 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:35.128487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d8b3f-478c-44dc-9afc-b26239cb6278 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:35.128785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d8b3f-478c-44dc-9afc-b26239cb6278 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:35.130385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d8b3f-478c-44dc-9afc-b26239cb6278 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:35.144976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 334/681] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:35.150080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d8b3f-478c-44dc-9afc-b26239cb6278 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:35.165537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:35.168565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 348/682] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:35.169808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414f553-b84b-45cd-8506-8ea576dbb7fe None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:35.283387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d8b3f-478c-44dc-9afc-b26239cb6278 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:35.299584 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:35.305642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414f553-b84b-45cd-8506-8ea576dbb7fe None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:35.325993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b88102-fac2-4805-969b-caf8503b6dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:35.326787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b88102-fac2-4805-969b-caf8503b6dd6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:35.327264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b88102-fac2-4805-969b-caf8503b6dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:35.327631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b88102-fac2-4805-969b-caf8503b6dd6 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:35.328376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b88102-fac2-4805-969b-caf8503b6dd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:35.328847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b88102-fac2-4805-969b-caf8503b6dd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:35.330279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b88102-fac2-4805-969b-caf8503b6dd6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:35.347075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 335/683] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:35.347075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b88102-fac2-4805-969b-caf8503b6dd6 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:35.355768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df4989-263f-4c77-9a8a-e32020c6c82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:35.357182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df4989-263f-4c77-9a8a-e32020c6c82e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:35.357658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df4989-263f-4c77-9a8a-e32020c6c82e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:35.358114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df4989-263f-4c77-9a8a-e32020c6c82e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:35.359003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df4989-263f-4c77-9a8a-e32020c6c82e None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=289e7caceab944e9b90f2e9298fb2cf1) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:35.359944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df4989-263f-4c77-9a8a-e32020c6c82e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:35.362117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df4989-263f-4c77-9a8a-e32020c6c82e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:35.376701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 349/684] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:35 2022] GET /identity/v3/role_assignments?user.id=289e7caceab944e9b90f2e9298fb2cf1&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:35.376701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df4989-263f-4c77-9a8a-e32020c6c82e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:35.471127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:35.527091 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b88102-fac2-4805-969b-caf8503b6dd6 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:35.578874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2eea1-aea1-4870-9d51-da55f90fac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:35.579438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2eea1-aea1-4870-9d51-da55f90fac65 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:35.579711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2eea1-aea1-4870-9d51-da55f90fac65 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:35.579975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2eea1-aea1-4870-9d51-da55f90fac65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:35.590130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df4989-263f-4c77-9a8a-e32020c6c82e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:35.593307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2eea1-aea1-4870-9d51-da55f90fac65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-2bb2eea1-aea1-4870-9d51-da55f90fac65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:35.600196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:35.601685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:35.602552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 336/685] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:35.608099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2eea1-aea1-4870-9d51-da55f90fac65 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:35.647817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3eec4-c8f0-44e9-920a-1ac3f2bfae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:35.649248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3eec4-c8f0-44e9-920a-1ac3f2bfae89 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:35.649720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3eec4-c8f0-44e9-920a-1ac3f2bfae89 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:35.650334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3eec4-c8f0-44e9-920a-1ac3f2bfae89 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:35.661665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3eec4-c8f0-44e9-920a-1ac3f2bfae89 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:35.666946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3eec4-c8f0-44e9-920a-1ac3f2bfae89 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-6cd3eec4-c8f0-44e9-920a-1ac3f2bfae89 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:35.684984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:35.685985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:35.686912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:35.686912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:35.686912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:35.686912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:35.686912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:35.686912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:35.686912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:35.686912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:35.686912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:35.686912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:35.689486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 350/686] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:35.696346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3eec4-c8f0-44e9-920a-1ac3f2bfae89 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:35.728866 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:35.754030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2eea1-aea1-4870-9d51-da55f90fac65 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:35.793913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:35.809459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a25b6-c13a-4b45-8f34-59621a9ab639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:35.810111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a25b6-c13a-4b45-8f34-59621a9ab639 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:35.810489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a25b6-c13a-4b45-8f34-59621a9ab639 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:35.810757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a25b6-c13a-4b45-8f34-59621a9ab639 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:35.812875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a25b6-c13a-4b45-8f34-59621a9ab639 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:35.812875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a25b6-c13a-4b45-8f34-59621a9ab639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:35.813077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a25b6-c13a-4b45-8f34-59621a9ab639 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:35.825301 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 337/687] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:35.830927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a25b6-c13a-4b45-8f34-59621a9ab639 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:35.845946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3eec4-c8f0-44e9-920a-1ac3f2bfae89 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:35.912506 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4068c-17de-425d-8cfa-5872cad8b7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:35.913104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4068c-17de-425d-8cfa-5872cad8b7c4 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:35.913897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4068c-17de-425d-8cfa-5872cad8b7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:35.914189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4068c-17de-425d-8cfa-5872cad8b7c4 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:35.914906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4068c-17de-425d-8cfa-5872cad8b7c4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:35.915200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4068c-17de-425d-8cfa-5872cad8b7c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:35.916613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4068c-17de-425d-8cfa-5872cad8b7c4 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:35.947412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 351/688] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Aug 9 16:32:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:35.948792 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623bb90-a74b-4f19-adcf-82a6a4844efd None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:35.949068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623bb90-a74b-4f19-adcf-82a6a4844efd None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:35.949333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2623bb90-a74b-4f19-adcf-82a6a4844efd None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:35.950892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 352/689] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:35.955519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623bb90-a74b-4f19-adcf-82a6a4844efd None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:35.973627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a25b6-c13a-4b45-8f34-59621a9ab639 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:36.016632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11978061-4f3a-4f95-b4c8-9b6312a14d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:36.017250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11978061-4f3a-4f95-b4c8-9b6312a14d63 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:36.017675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11978061-4f3a-4f95-b4c8-9b6312a14d63 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:36.017945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11978061-4f3a-4f95-b4c8-9b6312a14d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:36.031841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11978061-4f3a-4f95-b4c8-9b6312a14d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:36.033426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11978061-4f3a-4f95-b4c8-9b6312a14d63 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-11978061-4f3a-4f95-b4c8-9b6312a14d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:36.051386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:36.052405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:36.054857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:36.054857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:36.054857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:36.054857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 338/690] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:36.054857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4558d-5c3b-4841-a11a-bc030ad6f4cc None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:36.054857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4558d-5c3b-4841-a11a-bc030ad6f4cc None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:36.054857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4558d-5c3b-4841-a11a-bc030ad6f4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:36.055763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:36.078117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-89a4558d-5c3b-4841-a11a-bc030ad6f4cc None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:36.078431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2623bb90-a74b-4f19-adcf-82a6a4844efd None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:36.079318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-89a4558d-5c3b-4841-a11a-bc030ad6f4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:36.120653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e85a5-9b1c-416f-9cfa-03bc92f81a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:36.121174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e85a5-9b1c-416f-9cfa-03bc92f81a49 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:36.121451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e85a5-9b1c-416f-9cfa-03bc92f81a49 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:36.121717 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e85a5-9b1c-416f-9cfa-03bc92f81a49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:36.142945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:36.181274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 339/691] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:36.182981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4558d-5c3b-4841-a11a-bc030ad6f4cc None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:36.202518 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e85a5-9b1c-416f-9cfa-03bc92f81a49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:36.204031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e85a5-9b1c-416f-9cfa-03bc92f81a49 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:36.299154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:36.322170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 353/692] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:36.323486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e85a5-9b1c-416f-9cfa-03bc92f81a49 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:36.433013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e85a5-9b1c-416f-9cfa-03bc92f81a49 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:36.449843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4558d-5c3b-4841-a11a-bc030ad6f4cc None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:36.476919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b60cc6-0b94-486f-9c78-bd4d13e71577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:36.478148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60cc6-0b94-486f-9c78-bd4d13e71577 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:36.478854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60cc6-0b94-486f-9c78-bd4d13e71577 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:36.479303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60cc6-0b94-486f-9c78-bd4d13e71577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:36.487813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64351f8-7a5d-4ae5-8272-c7c38855de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:36.488606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64351f8-7a5d-4ae5-8272-c7c38855de31 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:36.488858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64351f8-7a5d-4ae5-8272-c7c38855de31 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:36.489219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64351f8-7a5d-4ae5-8272-c7c38855de31 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:36.489915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64351f8-7a5d-4ae5-8272-c7c38855de31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:36.490337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64351f8-7a5d-4ae5-8272-c7c38855de31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:36.491962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64351f8-7a5d-4ae5-8272-c7c38855de31 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:36.493572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b60cc6-0b94-486f-9c78-bd4d13e71577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:36.495836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b60cc6-0b94-486f-9c78-bd4d13e71577 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:36.505660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 340/693] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:36.505660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64351f8-7a5d-4ae5-8272-c7c38855de31 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-a9b60cc6-0b94-486f-9c78-bd4d13e71577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:36.518078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:36.519180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:36.520061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:36.520061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:36.520061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:36.520401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 354/694] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:36.521302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b60cc6-0b94-486f-9c78-bd4d13e71577 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:36.555354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:36.559728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:36.629883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b60cc6-0b94-486f-9c78-bd4d13e71577 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:36.650630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64351f8-7a5d-4ae5-8272-c7c38855de31 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:36.666652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc43f69-62fd-4c28-8937-b2ef33dad399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:36.667410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc43f69-62fd-4c28-8937-b2ef33dad399 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:36.667785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc43f69-62fd-4c28-8937-b2ef33dad399 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:36.668169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc43f69-62fd-4c28-8937-b2ef33dad399 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:36.676418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc43f69-62fd-4c28-8937-b2ef33dad399 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:36.678015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc43f69-62fd-4c28-8937-b2ef33dad399 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-abc43f69-62fd-4c28-8937-b2ef33dad399 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:36.690554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:36.691711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:36.692898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:36.692898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:36.692898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:36.692898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:36.692898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:36.692898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:36.692898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:36.692898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:36.692898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:36.692898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:36.699523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 355/695] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Aug 9 16:32:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:36.699523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc43f69-62fd-4c28-8937-b2ef33dad399 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:36.703012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a9ab6-54cb-43d6-8ebc-f1b7ceac8f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:36.703893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a9ab6-54cb-43d6-8ebc-f1b7ceac8f4f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:36.704334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a9ab6-54cb-43d6-8ebc-f1b7ceac8f4f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:36.704744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a9ab6-54cb-43d6-8ebc-f1b7ceac8f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:36.718772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a9ab6-54cb-43d6-8ebc-f1b7ceac8f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:36.722619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a9ab6-54cb-43d6-8ebc-f1b7ceac8f4f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-5b4a9ab6-54cb-43d6-8ebc-f1b7ceac8f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:36.739876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:36.741088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:36.742706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:36.742706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:36.742706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:36.742706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 341/696] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:36.748132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a9ab6-54cb-43d6-8ebc-f1b7ceac8f4f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:36.822372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:36.826517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc43f69-62fd-4c28-8937-b2ef33dad399 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:36.844823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:36.860910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a9ab6-54cb-43d6-8ebc-f1b7ceac8f4f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:36.873286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc78f7-bd17-4f14-88fd-329682c04ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:36.874170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc78f7-bd17-4f14-88fd-329682c04ca0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:36.874768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc78f7-bd17-4f14-88fd-329682c04ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:36.875654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc78f7-bd17-4f14-88fd-329682c04ca0 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:36.876779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc78f7-bd17-4f14-88fd-329682c04ca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:36.877267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc78f7-bd17-4f14-88fd-329682c04ca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:36.879289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc78f7-bd17-4f14-88fd-329682c04ca0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:36.896862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a4184-bb51-4500-a010-ee0d3414fd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:36.897470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a4184-bb51-4500-a010-ee0d3414fd29 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:36.897776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a4184-bb51-4500-a010-ee0d3414fd29 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:36.898086 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a4184-bb51-4500-a010-ee0d3414fd29 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:36.898813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a4184-bb51-4500-a010-ee0d3414fd29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:36.899109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a4184-bb51-4500-a010-ee0d3414fd29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:36.900610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a4184-bb51-4500-a010-ee0d3414fd29 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:36.931378 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 342/697] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Aug 9 16:32:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:36.932736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a4184-bb51-4500-a010-ee0d3414fd29 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:37.021068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 356/698] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:37.026792 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc78f7-bd17-4f14-88fd-329682c04ca0 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:37.052486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a4184-bb51-4500-a010-ee0d3414fd29 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:37.094794 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ecfa7-8192-4514-bb39-410d34c9483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:37.095366 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ecfa7-8192-4514-bb39-410d34c9483f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.095636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ecfa7-8192-4514-bb39-410d34c9483f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.095896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ecfa7-8192-4514-bb39-410d34c9483f None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.096517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ecfa7-8192-4514-bb39-410d34c9483f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:37.096810 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ecfa7-8192-4514-bb39-410d34c9483f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:37.098445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ecfa7-8192-4514-bb39-410d34c9483f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:37.117493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 343/699] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:37.117493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ecfa7-8192-4514-bb39-410d34c9483f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:37.120476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:37.138431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc78f7-bd17-4f14-88fd-329682c04ca0 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:37.174492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4446b73-3915-4efc-a336-4b9c37eb54c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:37.175089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4446b73-3915-4efc-a336-4b9c37eb54c2 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.175363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4446b73-3915-4efc-a336-4b9c37eb54c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.175634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4446b73-3915-4efc-a336-4b9c37eb54c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.197083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:37.222798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ecfa7-8192-4514-bb39-410d34c9483f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:37.244893 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4446b73-3915-4efc-a336-4b9c37eb54c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:37.246354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4446b73-3915-4efc-a336-4b9c37eb54c2 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:37.258379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf63d7f-97b3-41f9-9495-9d9960efe165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:37.259000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf63d7f-97b3-41f9-9495-9d9960efe165 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.259270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf63d7f-97b3-41f9-9495-9d9960efe165 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.259529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf63d7f-97b3-41f9-9495-9d9960efe165 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.260155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf63d7f-97b3-41f9-9495-9d9960efe165 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:37.261555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf63d7f-97b3-41f9-9495-9d9960efe165 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:37.343268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 357/700] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:37.344439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf31a96-f21c-4bfb-9501-89ff3a0926b5 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.344718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf31a96-f21c-4bfb-9501-89ff3a0926b5 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.344988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf31a96-f21c-4bfb-9501-89ff3a0926b5 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.350139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 358/701] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:37.353849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf31a96-f21c-4bfb-9501-89ff3a0926b5 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:37.379149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 344/702] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:32:37 2022] POST /identity/v3/users => generated 254 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:32:37.380246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf63d7f-97b3-41f9-9495-9d9960efe165 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:37.436434 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:37.468987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:37.501685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf31a96-f21c-4bfb-9501-89ff3a0926b5 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:37.533506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf63d7f-97b3-41f9-9495-9d9960efe165 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:37.553541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881ed95-873e-4a30-90a8-262eb8be3372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:37.554798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ed95-873e-4a30-90a8-262eb8be3372 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.555077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ed95-873e-4a30-90a8-262eb8be3372 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.555350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881ed95-873e-4a30-90a8-262eb8be3372 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.565681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881ed95-873e-4a30-90a8-262eb8be3372 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:37.567183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881ed95-873e-4a30-90a8-262eb8be3372 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-c881ed95-873e-4a30-90a8-262eb8be3372 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:37.582677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:37.583716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:37.584583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:37.584583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:37.584583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:37.584583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:37.584583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:37.584583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:37.584583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:37.584583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:37.585006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf25972-31a3-4380-aa50-78f272906216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:37.585006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf25972-31a3-4380-aa50-78f272906216 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.585006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf25972-31a3-4380-aa50-78f272906216 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.585006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf25972-31a3-4380-aa50-78f272906216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.586704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 359/703] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:37.588216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1880c5d-5a88-42cb-97a1-005b33562399 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.589103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1880c5d-5a88-42cb-97a1-005b33562399 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.589608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1880c5d-5a88-42cb-97a1-005b33562399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.597995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf25972-31a3-4380-aa50-78f272906216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:37.599455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf25972-31a3-4380-aa50-78f272906216 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:37.611678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a1880c5d-5a88-42cb-97a1-005b33562399 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:37.612882 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a1880c5d-5a88-42cb-97a1-005b33562399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-ccf25972-31a3-4380-aa50-78f272906216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:37.614462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:37.618857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:37.619737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:37.619737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:37.619737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:37.619737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 345/704] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:37.619737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf25972-31a3-4380-aa50-78f272906216 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:37.696463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 360/705] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:37.698001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1880c5d-5a88-42cb-97a1-005b33562399 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:37.753277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf25972-31a3-4380-aa50-78f272906216 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:37.765623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:37.771216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:37.793074 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cc65a-0723-4810-b274-7df3582ee303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:37.793669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cc65a-0723-4810-b274-7df3582ee303 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.793939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cc65a-0723-4810-b274-7df3582ee303 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.794228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cc65a-0723-4810-b274-7df3582ee303 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.794909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cc65a-0723-4810-b274-7df3582ee303 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:37.795203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cc65a-0723-4810-b274-7df3582ee303 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:37.796648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cc65a-0723-4810-b274-7df3582ee303 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:37.810928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 346/706] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:37.811640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cc65a-0723-4810-b274-7df3582ee303 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:37.832957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1880c5d-5a88-42cb-97a1-005b33562399 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:37.869795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c8d1b-41a1-478b-994a-ef4fae1e0a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:37.870407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c8d1b-41a1-478b-994a-ef4fae1e0a49 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.870680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c8d1b-41a1-478b-994a-ef4fae1e0a49 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.870948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c8d1b-41a1-478b-994a-ef4fae1e0a49 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.871611 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c8d1b-41a1-478b-994a-ef4fae1e0a49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:37.871907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c8d1b-41a1-478b-994a-ef4fae1e0a49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:37.873256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72c8d1b-41a1-478b-994a-ef4fae1e0a49 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:37.885991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 361/707] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:37.885991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b9d3f-c223-41e4-b252-6a32d9ce2359 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.886199 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b9d3f-c223-41e4-b252-6a32d9ce2359 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.886547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b9d3f-c223-41e4-b252-6a32d9ce2359 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.887631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 362/708] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:37.891810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b9d3f-c223-41e4-b252-6a32d9ce2359 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:37.921065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cc65a-0723-4810-b274-7df3582ee303 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:37.960309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83b0f4-f3a5-4ea1-94bf-177660a43bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:37.960936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83b0f4-f3a5-4ea1-94bf-177660a43bf8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:37.961209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83b0f4-f3a5-4ea1-94bf-177660a43bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:37.961474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83b0f4-f3a5-4ea1-94bf-177660a43bf8 None admin] PATH_INFO: `/v3/services/network` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:37.962197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83b0f4-f3a5-4ea1-94bf-177660a43bf8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:37.963652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83b0f4-f3a5-4ea1-94bf-177660a43bf8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-9c83b0f4-f3a5-4ea1-94bf-177660a43bf8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:37.974857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:37.976458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:37.977313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 347/709] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:37 2022] GET /identity/v3/services/network => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:37.978810 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83b0f4-f3a5-4ea1-94bf-177660a43bf8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:38.005569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b9d3f-c223-41e4-b252-6a32d9ce2359 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:38.007141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:38.017726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:38.039935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e31a8-8b97-418c-bdac-db982600c9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:38.040797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e31a8-8b97-418c-bdac-db982600c9ab None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:38.041229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e31a8-8b97-418c-bdac-db982600c9ab None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:38.041516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e31a8-8b97-418c-bdac-db982600c9ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:38.042235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e31a8-8b97-418c-bdac-db982600c9ab None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=35371b488c474b7f851af16bdad34f69) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:38.042536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e31a8-8b97-418c-bdac-db982600c9ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:38.043944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e31a8-8b97-418c-bdac-db982600c9ab None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:38.062051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 363/710] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:37 2022] GET /identity/v3/role_assignments?user.id=35371b488c474b7f851af16bdad34f69&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:38.062051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41113c5-62e8-4e65-879c-3ae677a0c749 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:38.062051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41113c5-62e8-4e65-879c-3ae677a0c749 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:38.062051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41113c5-62e8-4e65-879c-3ae677a0c749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:38.083731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a41113c5-62e8-4e65-879c-3ae677a0c749 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:38.085501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a41113c5-62e8-4e65-879c-3ae677a0c749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:38.106217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83b0f4-f3a5-4ea1-94bf-177660a43bf8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:38.147464 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e850e-2306-4e71-b5b6-d468883efe5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:38.148199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e850e-2306-4e71-b5b6-d468883efe5c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:38.148540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e850e-2306-4e71-b5b6-d468883efe5c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:38.148869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e850e-2306-4e71-b5b6-d468883efe5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:38.157432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e850e-2306-4e71-b5b6-d468883efe5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:38.159004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e850e-2306-4e71-b5b6-d468883efe5c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-8f4e850e-2306-4e71-b5b6-d468883efe5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:38.172486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:38.173468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:38.174642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:38.174642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:38.174642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:38.174642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:38.174642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:38.174642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:38.174642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:38.174642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:38.174642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 348/712] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:38.175133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 364/712] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:38.175219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e850e-2306-4e71-b5b6-d468883efe5c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:38.181640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41113c5-62e8-4e65-879c-3ae677a0c749 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:38.289687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:38.295834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e850e-2306-4e71-b5b6-d468883efe5c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:38.309018 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:38.317151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41113c5-62e8-4e65-879c-3ae677a0c749 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:38.336842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c16332-245f-4bd0-8b78-eb8e8ee14f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:38.337448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c16332-245f-4bd0-8b78-eb8e8ee14f0b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:38.337725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c16332-245f-4bd0-8b78-eb8e8ee14f0b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:38.338634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c16332-245f-4bd0-8b78-eb8e8ee14f0b None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:38.338634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c16332-245f-4bd0-8b78-eb8e8ee14f0b None admin] RBAC: Adding query filter params (name=network) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:38.344299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c16332-245f-4bd0-8b78-eb8e8ee14f0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:38.344299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c16332-245f-4bd0-8b78-eb8e8ee14f0b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:38.349327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 349/713] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:32:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:38.351786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c16332-245f-4bd0-8b78-eb8e8ee14f0b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:38.367475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af782430-e3de-4c55-a37f-ce2b1fd54d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:38.368365 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af782430-e3de-4c55-a37f-ce2b1fd54d52 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:38.368816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af782430-e3de-4c55-a37f-ce2b1fd54d52 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:38.369253 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af782430-e3de-4c55-a37f-ce2b1fd54d52 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:38.370188 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af782430-e3de-4c55-a37f-ce2b1fd54d52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:38.370665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af782430-e3de-4c55-a37f-ce2b1fd54d52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:38.372272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af782430-e3de-4c55-a37f-ce2b1fd54d52 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:38.387966 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 365/714] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:38.388855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af782430-e3de-4c55-a37f-ce2b1fd54d52 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:38.471944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c16332-245f-4bd0-8b78-eb8e8ee14f0b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:38.505330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af782430-e3de-4c55-a37f-ce2b1fd54d52 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:38.507572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2414fd-e004-4ef5-83ac-847bcde3defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:38.508396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2414fd-e004-4ef5-83ac-847bcde3defa None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:38.508843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2414fd-e004-4ef5-83ac-847bcde3defa None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:38.509341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2414fd-e004-4ef5-83ac-847bcde3defa None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:38.510178 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2414fd-e004-4ef5-83ac-847bcde3defa None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:38.511713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2414fd-e004-4ef5-83ac-847bcde3defa None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-1a2414fd-e004-4ef5-83ac-847bcde3defa None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:38.520921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:38.521916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 16:32:38.523347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:38.524097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 350/715] 10.222.0.79 () {56 vars in 1129 bytes} [Tue Aug 9 16:32:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:38.524756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2414fd-e004-4ef5-83ac-847bcde3defa None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:38.545872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ea483-722d-4e27-9ba9-986ac3425281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:38.546515 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ea483-722d-4e27-9ba9-986ac3425281 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:38.546793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ea483-722d-4e27-9ba9-986ac3425281 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:38.547063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ea483-722d-4e27-9ba9-986ac3425281 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:38.547759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ea483-722d-4e27-9ba9-986ac3425281 None admin] RBAC: Adding query filter params (type=network) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:38.548063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ea483-722d-4e27-9ba9-986ac3425281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:38.549616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ea483-722d-4e27-9ba9-986ac3425281 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:38.556297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 366/716] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:32:38 2022] GET /identity/v3/services?type=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:38.557080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ea483-722d-4e27-9ba9-986ac3425281 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:38.569014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:38.583853 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:38.732621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2414fd-e004-4ef5-83ac-847bcde3defa None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:38.759120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ea483-722d-4e27-9ba9-986ac3425281 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:38.804759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0bdc8-f6bb-4e19-888a-71df6ad2a1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:38.805374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0bdc8-f6bb-4e19-888a-71df6ad2a1fb None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:38.805650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0bdc8-f6bb-4e19-888a-71df6ad2a1fb None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:38.805919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0bdc8-f6bb-4e19-888a-71df6ad2a1fb None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:38.806090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e942f6-8674-4874-bb2a-131018b9a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:38.806588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0bdc8-f6bb-4e19-888a-71df6ad2a1fb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:38.806891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0bdc8-f6bb-4e19-888a-71df6ad2a1fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:38.807362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e942f6-8674-4874-bb2a-131018b9a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:38.808061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e942f6-8674-4874-bb2a-131018b9a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:38.808249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0bdc8-f6bb-4e19-888a-71df6ad2a1fb None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:38.808902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e942f6-8674-4874-bb2a-131018b9a6d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:38.810730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e942f6-8674-4874-bb2a-131018b9a6d7 None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=5d17aa4b23e04290b6c8b1924b0dcf32) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:38.811967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e942f6-8674-4874-bb2a-131018b9a6d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:38.814910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 367/717] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Aug 9 16:32:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:38.815463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e942f6-8674-4874-bb2a-131018b9a6d7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:38.820489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0bdc8-f6bb-4e19-888a-71df6ad2a1fb None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:38.824598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 351/718] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:38 2022] GET /identity/v3/role_assignments?user.id=5d17aa4b23e04290b6c8b1924b0dcf32&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 259 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:38.916908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:38.935667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0bdc8-f6bb-4e19-888a-71df6ad2a1fb None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:38.974277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bca154-8c02-46d5-bcee-b8d06d5cc7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:38.974883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bca154-8c02-46d5-bcee-b8d06d5cc7d9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:38.975157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bca154-8c02-46d5-bcee-b8d06d5cc7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:38.975423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bca154-8c02-46d5-bcee-b8d06d5cc7d9 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:38.976629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bca154-8c02-46d5-bcee-b8d06d5cc7d9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:38.976924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bca154-8c02-46d5-bcee-b8d06d5cc7d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:38.981831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bca154-8c02-46d5-bcee-b8d06d5cc7d9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:38.986113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 368/719] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Aug 9 16:32:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:39.639605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4b02a-e71a-4ac0-97bc-24d005b3b5dc None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:39.640151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4b02a-e71a-4ac0-97bc-24d005b3b5dc None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:39.640151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4b02a-e71a-4ac0-97bc-24d005b3b5dc None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:39.641716 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 352/720] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:39.646951 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3eef8-ee6f-444d-b7e3-b0e6c9a028d0 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:39.647255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3eef8-ee6f-444d-b7e3-b0e6c9a028d0 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:39.647527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3eef8-ee6f-444d-b7e3-b0e6c9a028d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:39.666392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5de3eef8-ee6f-444d-b7e3-b0e6c9a028d0 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:39.667565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5de3eef8-ee6f-444d-b7e3-b0e6c9a028d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:39.736673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 369/721] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:39.760742 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4b02a-e71a-4ac0-97bc-24d005b3b5dc None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:39.869875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:39.891629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4b02a-e71a-4ac0-97bc-24d005b3b5dc None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:39.942671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3baeca-1a49-4207-ab24-3f401123e531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:39.943313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3baeca-1a49-4207-ab24-3f401123e531 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:39.943591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3baeca-1a49-4207-ab24-3f401123e531 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:39.943860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3baeca-1a49-4207-ab24-3f401123e531 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:39.959404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3baeca-1a49-4207-ab24-3f401123e531 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-5f3baeca-1a49-4207-ab24-3f401123e531 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:39.964072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:39.965214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:39.966507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 353/722] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:39.971481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3eef8-ee6f-444d-b7e3-b0e6c9a028d0 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:40.077709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:40.105352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3eef8-ee6f-444d-b7e3-b0e6c9a028d0 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:40.156591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67353588-19c4-46cb-b0dc-a15760f15b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:40.157116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67353588-19c4-46cb-b0dc-a15760f15b1d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:40.157390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67353588-19c4-46cb-b0dc-a15760f15b1d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:40.157657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67353588-19c4-46cb-b0dc-a15760f15b1d None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:40.158354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67353588-19c4-46cb-b0dc-a15760f15b1d None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:40.158651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67353588-19c4-46cb-b0dc-a15760f15b1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:40.160014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67353588-19c4-46cb-b0dc-a15760f15b1d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:40.173502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 370/723] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:40.178708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3baeca-1a49-4207-ab24-3f401123e531 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:40.324974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3baeca-1a49-4207-ab24-3f401123e531 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:40.368165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0a409-5d19-486e-a505-1c85b046f715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:40.369046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0a409-5d19-486e-a505-1c85b046f715 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:40.369481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0a409-5d19-486e-a505-1c85b046f715 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:40.369885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0a409-5d19-486e-a505-1c85b046f715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:40.387675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0a409-5d19-486e-a505-1c85b046f715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:40.389389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0a409-5d19-486e-a505-1c85b046f715 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:40.401293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-8aa0a409-5d19-486e-a505-1c85b046f715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:40.415972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:40.417149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:40.418329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:40.418329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:40.418329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:40.418329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 354/724] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:40.418618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67353588-19c4-46cb-b0dc-a15760f15b1d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:40.537977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67353588-19c4-46cb-b0dc-a15760f15b1d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:40.568655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02c6bb-1257-483a-a18f-167c1f214c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:40.569250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02c6bb-1257-483a-a18f-167c1f214c8f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:40.569549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02c6bb-1257-483a-a18f-167c1f214c8f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:40.570350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02c6bb-1257-483a-a18f-167c1f214c8f None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:40.570988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02c6bb-1257-483a-a18f-167c1f214c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:40.571280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02c6bb-1257-483a-a18f-167c1f214c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:40.572633 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02c6bb-1257-483a-a18f-167c1f214c8f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:40.582821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:40.588044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 371/725] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:40.593853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0a409-5d19-486e-a505-1c85b046f715 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:40.614582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a23e7-b247-4e45-932b-4caee22f95f0 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:40.614910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a23e7-b247-4e45-932b-4caee22f95f0 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:40.615134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a23e7-b247-4e45-932b-4caee22f95f0 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:40.616830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 372/726] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:40.621851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303c7c-84b0-415a-929e-e44d12f82027 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:40.622161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303c7c-84b0-415a-929e-e44d12f82027 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:40.622501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df303c7c-84b0-415a-929e-e44d12f82027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:40.650987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-df303c7c-84b0-415a-929e-e44d12f82027 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:40.652537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-df303c7c-84b0-415a-929e-e44d12f82027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:40.733987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0a409-5d19-486e-a505-1c85b046f715 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:40.769456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be3fa-d5b4-4849-826a-243a4a6d6646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:40.770111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be3fa-d5b4-4849-826a-243a4a6d6646 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:40.770380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be3fa-d5b4-4849-826a-243a4a6d6646 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:40.771011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be3fa-d5b4-4849-826a-243a4a6d6646 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:40.775304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 373/727] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:40.786004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1be3fa-d5b4-4849-826a-243a4a6d6646 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:40.786004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1be3fa-d5b4-4849-826a-243a4a6d6646 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-0d1be3fa-d5b4-4849-826a-243a4a6d6646 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:40.792713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:40.793648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:40.794939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 355/728] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Aug 9 16:32:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:40.803204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303c7c-84b0-415a-929e-e44d12f82027 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:40.812210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be3fa-d5b4-4849-826a-243a4a6d6646 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:40.879621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:40.934756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be3fa-d5b4-4849-826a-243a4a6d6646 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:40.973065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c5160-6991-4f58-b40c-7d6c89de8dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:40.974608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c5160-6991-4f58-b40c-7d6c89de8dd8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:40.975092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c5160-6991-4f58-b40c-7d6c89de8dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:40.975556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c5160-6991-4f58-b40c-7d6c89de8dd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:40.991801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c5160-6991-4f58-b40c-7d6c89de8dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-cf1c5160-6991-4f58-b40c-7d6c89de8dd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:40.996130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:40.997050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:40.997865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:41.002466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 356/729] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:41.008413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c5160-6991-4f58-b40c-7d6c89de8dd8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:41.059851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df303c7c-84b0-415a-929e-e44d12f82027 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:41.118191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:41.127295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f984a5-7613-4b8c-b75e-738902186a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:41.128265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f984a5-7613-4b8c-b75e-738902186a8a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.128747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f984a5-7613-4b8c-b75e-738902186a8a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.130964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f984a5-7613-4b8c-b75e-738902186a8a None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.130964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f984a5-7613-4b8c-b75e-738902186a8a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:41.130964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f984a5-7613-4b8c-b75e-738902186a8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:41.132749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f984a5-7613-4b8c-b75e-738902186a8a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:41.145941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c5160-6991-4f58-b40c-7d6c89de8dd8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:41.157577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 374/730] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Aug 9 16:32:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:41.165360 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f984a5-7613-4b8c-b75e-738902186a8a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:41.177487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:41.193833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c45cc2-3c25-496e-b782-2b18f05b0157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:41.194541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c45cc2-3c25-496e-b782-2b18f05b0157 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.195339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c45cc2-3c25-496e-b782-2b18f05b0157 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.195608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c45cc2-3c25-496e-b782-2b18f05b0157 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.197470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c45cc2-3c25-496e-b782-2b18f05b0157 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:41.197848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c45cc2-3c25-496e-b782-2b18f05b0157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:41.199239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c45cc2-3c25-496e-b782-2b18f05b0157 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:41.223881 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 357/731] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:41.227322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c45cc2-3c25-496e-b782-2b18f05b0157 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:41.317029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f984a5-7613-4b8c-b75e-738902186a8a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:41.359650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c45cc2-3c25-496e-b782-2b18f05b0157 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:41.362871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e628356-bd86-456b-8f85-bf0565b9ac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:41.364399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e628356-bd86-456b-8f85-bf0565b9ac07 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.364691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e628356-bd86-456b-8f85-bf0565b9ac07 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.364963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e628356-bd86-456b-8f85-bf0565b9ac07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.410160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34838dfc-199b-4a7c-b238-2e217e89eb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:41.411151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34838dfc-199b-4a7c-b238-2e217e89eb12 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.411644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34838dfc-199b-4a7c-b238-2e217e89eb12 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.412113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34838dfc-199b-4a7c-b238-2e217e89eb12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.426423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34838dfc-199b-4a7c-b238-2e217e89eb12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:41.428770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34838dfc-199b-4a7c-b238-2e217e89eb12 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:41.432004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:41.446799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e628356-bd86-456b-8f85-bf0565b9ac07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:41.448338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e628356-bd86-456b-8f85-bf0565b9ac07 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-34838dfc-199b-4a7c-b238-2e217e89eb12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:41.452106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:41.453053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:41.453928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:41.453928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:41.453928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:41.457346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 358/732] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:41.458930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5befd3a-1229-4b60-87dc-45dd0ef8a992 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.459448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5befd3a-1229-4b60-87dc-45dd0ef8a992 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.459940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5befd3a-1229-4b60-87dc-45dd0ef8a992 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.466184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 359/733] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:41.468393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5befd3a-1229-4b60-87dc-45dd0ef8a992 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:41.511735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:41.568620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 375/734] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 09 16:32:41.569804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6eabf1-9d5f-4f67-b2a5-9d1c1774421e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.570117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6eabf1-9d5f-4f67-b2a5-9d1c1774421e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.570382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6eabf1-9d5f-4f67-b2a5-9d1c1774421e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.585980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5befd3a-1229-4b60-87dc-45dd0ef8a992 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:41.589342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-4e6eabf1-9d5f-4f67-b2a5-9d1c1774421e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:41.590533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4e6eabf1-9d5f-4f67-b2a5-9d1c1774421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:41.624440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf130e16-0773-431c-a8b0-d428bbec0a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:41.625038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf130e16-0773-431c-a8b0-d428bbec0a8a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.625313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf130e16-0773-431c-a8b0-d428bbec0a8a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.625580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf130e16-0773-431c-a8b0-d428bbec0a8a None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.626257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf130e16-0773-431c-a8b0-d428bbec0a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:41.626557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf130e16-0773-431c-a8b0-d428bbec0a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:41.627910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf130e16-0773-431c-a8b0-d428bbec0a8a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:41.639968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 360/735] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:41.641225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35f9ec-e88e-43ff-9b18-29763ff1031a None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.641507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35f9ec-e88e-43ff-9b18-29763ff1031a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.641824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35f9ec-e88e-43ff-9b18-29763ff1031a None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.647237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 361/736] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:41.647237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35f9ec-e88e-43ff-9b18-29763ff1031a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:41.694060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 376/737] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:41.694060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6eabf1-9d5f-4f67-b2a5-9d1c1774421e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:41.712314 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:41.759271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:41.767118 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35f9ec-e88e-43ff-9b18-29763ff1031a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:41.792420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6eabf1-9d5f-4f67-b2a5-9d1c1774421e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:41.800488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f18e1-fb81-4ee8-b46e-b0bc03f76d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:41.801099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f18e1-fb81-4ee8-b46e-b0bc03f76d16 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.801374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f18e1-fb81-4ee8-b46e-b0bc03f76d16 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.801647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f18e1-fb81-4ee8-b46e-b0bc03f76d16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.812044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f18e1-fb81-4ee8-b46e-b0bc03f76d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:41.813462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43f18e1-fb81-4ee8-b46e-b0bc03f76d16 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:41.826350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff3d34-2b15-410f-a5fa-9344342ed156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:41.826891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff3d34-2b15-410f-a5fa-9344342ed156 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.827162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff3d34-2b15-410f-a5fa-9344342ed156 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.827424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff3d34-2b15-410f-a5fa-9344342ed156 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-b43f18e1-fb81-4ee8-b46e-b0bc03f76d16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:41.832002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:41.833060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:41.833996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:41.833996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:41.833996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:41.833996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 362/738] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:41.834226 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849895b5-9b19-4fe2-8179-4e705fd28280 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.834506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849895b5-9b19-4fe2-8179-4e705fd28280 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.834774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849895b5-9b19-4fe2-8179-4e705fd28280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.835842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff3d34-2b15-410f-a5fa-9344342ed156 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:41.837395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff3d34-2b15-410f-a5fa-9344342ed156 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-4cff3d34-2b15-410f-a5fa-9344342ed156 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:41.849221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:41.850210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:41.851632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 377/739] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:41.852408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff3d34-2b15-410f-a5fa-9344342ed156 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:41.855115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-849895b5-9b19-4fe2-8179-4e705fd28280 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:41.856529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-849895b5-9b19-4fe2-8179-4e705fd28280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:41.937137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 363/740] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:41.938583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21b7f-fe66-4f74-9448-6dea6ff3e1bd None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:41.938866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21b7f-fe66-4f74-9448-6dea6ff3e1bd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:41.939153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe21b7f-fe66-4f74-9448-6dea6ff3e1bd None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:41.940355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 364/741] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:41.941134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe21b7f-fe66-4f74-9448-6dea6ff3e1bd None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:41.950526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:41.974252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff3d34-2b15-410f-a5fa-9344342ed156 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:42.010136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637e2ff-276d-4aad-9b43-c82faa910bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:42.010728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637e2ff-276d-4aad-9b43-c82faa910bb2 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:42.011010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637e2ff-276d-4aad-9b43-c82faa910bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:42.011273 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637e2ff-276d-4aad-9b43-c82faa910bb2 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:42.011912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637e2ff-276d-4aad-9b43-c82faa910bb2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:42.013265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637e2ff-276d-4aad-9b43-c82faa910bb2 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:42.029649 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:42.052942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe21b7f-fe66-4f74-9448-6dea6ff3e1bd None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:42.084791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e5917-90af-4fb3-8e96-23a20e61b0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:42.085821 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e5917-90af-4fb3-8e96-23a20e61b0f1 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:42.086127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e5917-90af-4fb3-8e96-23a20e61b0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:42.086392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e5917-90af-4fb3-8e96-23a20e61b0f1 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:42.087018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e5917-90af-4fb3-8e96-23a20e61b0f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:42.087376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e5917-90af-4fb3-8e96-23a20e61b0f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:42.088737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e5917-90af-4fb3-8e96-23a20e61b0f1 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:42.094851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 378/742] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:32:41 2022] POST /identity/v3/services => generated 240 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:32:42.095678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637e2ff-276d-4aad-9b43-c82faa910bb2 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:42.100542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 365/743] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:42.102499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345f39c-bf74-4161-aaa1-10e34b1c8def None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:42.102785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345f39c-bf74-4161-aaa1-10e34b1c8def None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:42.103056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345f39c-bf74-4161-aaa1-10e34b1c8def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:42.123805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c345f39c-bf74-4161-aaa1-10e34b1c8def None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:42.125049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c345f39c-bf74-4161-aaa1-10e34b1c8def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:42.226484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 366/744] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:42 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 09 16:32:42.226484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345f39c-bf74-4161-aaa1-10e34b1c8def None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:42.244846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637e2ff-276d-4aad-9b43-c82faa910bb2 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:42.283392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:42.287114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcabf4-8f42-4e7c-8b2c-235148e0cb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:42.287893 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcabf4-8f42-4e7c-8b2c-235148e0cb5d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:42.288168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcabf4-8f42-4e7c-8b2c-235148e0cb5d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:42.292855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcabf4-8f42-4e7c-8b2c-235148e0cb5d None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:42.292855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcabf4-8f42-4e7c-8b2c-235148e0cb5d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:42.292855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcabf4-8f42-4e7c-8b2c-235148e0cb5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:42.292855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcabf4-8f42-4e7c-8b2c-235148e0cb5d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:42.301338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:42.313997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 379/745] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Aug 9 16:32:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:42.313997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcabf4-8f42-4e7c-8b2c-235148e0cb5d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:42.329404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345f39c-bf74-4161-aaa1-10e34b1c8def None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:42.364407 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65232b-b7fa-442c-8fdc-d690810364f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:42.364942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65232b-b7fa-442c-8fdc-d690810364f7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:42.365220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65232b-b7fa-442c-8fdc-d690810364f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:42.365486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65232b-b7fa-442c-8fdc-d690810364f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:42.377286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65232b-b7fa-442c-8fdc-d690810364f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-4f65232b-b7fa-442c-8fdc-d690810364f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:42.383070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:42.384213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:42.385073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 367/746] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:42.385782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65232b-b7fa-442c-8fdc-d690810364f7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:42.434838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcabf4-8f42-4e7c-8b2c-235148e0cb5d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:42.474355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e259f-59cf-4806-a510-45a3b9abe7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:42.474571 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e259f-59cf-4806-a510-45a3b9abe7a8 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:42.474828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e259f-59cf-4806-a510-45a3b9abe7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:42.475242 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323e259f-59cf-4806-a510-45a3b9abe7a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:42.482484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e259f-59cf-4806-a510-45a3b9abe7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:42.483944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323e259f-59cf-4806-a510-45a3b9abe7a8 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:42.494391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65232b-b7fa-442c-8fdc-d690810364f7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-323e259f-59cf-4806-a510-45a3b9abe7a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:42.495050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:42.495894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:42.496786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:42.496786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:42.496786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:42.496786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:42.496786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:42.496786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:42.496786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:42.496786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:42.500876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 380/747] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:42.501704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e259f-59cf-4806-a510-45a3b9abe7a8 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:42.521546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:42.540491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50c2a8-53e2-4fc4-ae02-e25512574df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:42.541092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50c2a8-53e2-4fc4-ae02-e25512574df2 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:42.541368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50c2a8-53e2-4fc4-ae02-e25512574df2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:42.541664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50c2a8-53e2-4fc4-ae02-e25512574df2 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:42.542378 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50c2a8-53e2-4fc4-ae02-e25512574df2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:42.544168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50c2a8-53e2-4fc4-ae02-e25512574df2 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:42.564344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:42.607693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323e259f-59cf-4806-a510-45a3b9abe7a8 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:42.640789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414200b-f3a3-4b70-a72c-05f250b90128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:42.641304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414200b-f3a3-4b70-a72c-05f250b90128 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:42.641577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414200b-f3a3-4b70-a72c-05f250b90128 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:42.641843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414200b-f3a3-4b70-a72c-05f250b90128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:42.662248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 368/748] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:32:42 2022] POST /identity/v3/services => generated 251 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:32:42.663252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50c2a8-53e2-4fc4-ae02-e25512574df2 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:42.733019 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414200b-f3a3-4b70-a72c-05f250b90128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:42.735003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414200b-f3a3-4b70-a72c-05f250b90128 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:42.806772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:42.832867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50c2a8-53e2-4fc4-ae02-e25512574df2 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:42.859923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 381/749] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:42 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 09 16:32:42.860774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414200b-f3a3-4b70-a72c-05f250b90128 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:42.878705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420f822-58da-40ca-bcf9-9caff75bd9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:42.879324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420f822-58da-40ca-bcf9-9caff75bd9f9 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:42.879603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420f822-58da-40ca-bcf9-9caff75bd9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:42.879874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420f822-58da-40ca-bcf9-9caff75bd9f9 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:42.880605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420f822-58da-40ca-bcf9-9caff75bd9f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:42.881024 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420f822-58da-40ca-bcf9-9caff75bd9f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:42.883094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420f822-58da-40ca-bcf9-9caff75bd9f9 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:42.892505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 369/750] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:42.893072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420f822-58da-40ca-bcf9-9caff75bd9f9 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:42.937251 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:42.992133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414200b-f3a3-4b70-a72c-05f250b90128 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:43.000313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420f822-58da-40ca-bcf9-9caff75bd9f9 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:43.038096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21244962-e916-4ec4-b33f-18cfcd676c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:43.038368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9f5f6-c418-4f3f-9ebe-98e57c9513ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:43.039330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9f5f6-c418-4f3f-9ebe-98e57c9513ea None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:43.039446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21244962-e916-4ec4-b33f-18cfcd676c19 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:43.039603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21244962-e916-4ec4-b33f-18cfcd676c19 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:43.039708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9f5f6-c418-4f3f-9ebe-98e57c9513ea None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:43.039854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21244962-e916-4ec4-b33f-18cfcd676c19 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:43.039952 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9f5f6-c418-4f3f-9ebe-98e57c9513ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:43.040607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21244962-e916-4ec4-b33f-18cfcd676c19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:43.040854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21244962-e916-4ec4-b33f-18cfcd676c19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:43.043003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21244962-e916-4ec4-b33f-18cfcd676c19 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:43.050688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9f5f6-c418-4f3f-9ebe-98e57c9513ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:43.052258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9f5f6-c418-4f3f-9ebe-98e57c9513ea None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:43.057388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 382/751] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:43.058323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21244962-e916-4ec4-b33f-18cfcd676c19 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-31c9f5f6-c418-4f3f-9ebe-98e57c9513ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:43.067866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:43.069084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:43.071078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:43.071078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:43.071078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:43.071078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 370/752] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:43.071078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9f5f6-c418-4f3f-9ebe-98e57c9513ea None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:43.081490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:43.176179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:43.185049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9f5f6-c418-4f3f-9ebe-98e57c9513ea None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:43.203506 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21244962-e916-4ec4-b33f-18cfcd676c19 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:43.225565 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ab262-21ed-4007-9e3a-633fba3eec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:43.226728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ab262-21ed-4007-9e3a-633fba3eec28 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:43.227482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ab262-21ed-4007-9e3a-633fba3eec28 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:43.227925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ab262-21ed-4007-9e3a-633fba3eec28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:43.228789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ab262-21ed-4007-9e3a-633fba3eec28 None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=a010ea3529644fca951a7eef28dbe97e) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:43.229295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ab262-21ed-4007-9e3a-633fba3eec28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:43.231456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ab262-21ed-4007-9e3a-633fba3eec28 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:43.239804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 371/753] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:43 2022] GET /identity/v3/role_assignments?user.id=a010ea3529644fca951a7eef28dbe97e&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:43.245982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ab262-21ed-4007-9e3a-633fba3eec28 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:43.259788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d081c8-13cc-4966-bf6c-3f32ed6d1c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:43.260806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d081c8-13cc-4966-bf6c-3f32ed6d1c00 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:43.261299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d081c8-13cc-4966-bf6c-3f32ed6d1c00 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:43.261798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d081c8-13cc-4966-bf6c-3f32ed6d1c00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:43.295217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d081c8-13cc-4966-bf6c-3f32ed6d1c00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:43.297083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d081c8-13cc-4966-bf6c-3f32ed6d1c00 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-98d081c8-13cc-4966-bf6c-3f32ed6d1c00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:43.331284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:43.332287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:43.333146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:43.333146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:43.333146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:43.342462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 383/754] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:43.347742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d081c8-13cc-4966-bf6c-3f32ed6d1c00 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:43.371579 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ab262-21ed-4007-9e3a-633fba3eec28 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:43.413846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6290fe-9769-4e66-9097-6f267627b91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:43.414469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6290fe-9769-4e66-9097-6f267627b91f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:43.414744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6290fe-9769-4e66-9097-6f267627b91f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:43.415010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6290fe-9769-4e66-9097-6f267627b91f None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:43.415938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6290fe-9769-4e66-9097-6f267627b91f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:43.416238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6290fe-9769-4e66-9097-6f267627b91f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:43.418136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6290fe-9769-4e66-9097-6f267627b91f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:43.434563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 372/755] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:43.439998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6290fe-9769-4e66-9097-6f267627b91f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:43.462554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:43.474299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:43.518288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d081c8-13cc-4966-bf6c-3f32ed6d1c00 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:43.568459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec85d2-1ee8-4a55-ae96-cec85cd3038a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:43.569166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec85d2-1ee8-4a55-ae96-cec85cd3038a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:43.569444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec85d2-1ee8-4a55-ae96-cec85cd3038a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:43.569746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cec85d2-1ee8-4a55-ae96-cec85cd3038a None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:43.570543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec85d2-1ee8-4a55-ae96-cec85cd3038a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:43.570948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec85d2-1ee8-4a55-ae96-cec85cd3038a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:43.572416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cec85d2-1ee8-4a55-ae96-cec85cd3038a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:43.577721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6290fe-9769-4e66-9097-6f267627b91f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:43.598384 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 384/756] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:43.615180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec85d2-1ee8-4a55-ae96-cec85cd3038a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:43.623030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549d6d3-5930-4236-bae8-a9ed08b66b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:43.623664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549d6d3-5930-4236-bae8-a9ed08b66b5a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:43.623942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549d6d3-5930-4236-bae8-a9ed08b66b5a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:43.624205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549d6d3-5930-4236-bae8-a9ed08b66b5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:43.638695 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549d6d3-5930-4236-bae8-a9ed08b66b5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:43.640158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549d6d3-5930-4236-bae8-a9ed08b66b5a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-8549d6d3-5930-4236-bae8-a9ed08b66b5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:43.660247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:43.661164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:43.662070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:43.662070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:43.662070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:43.662070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:43.662070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:43.662070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:43.662070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:43.662070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:43.667770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 373/757] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:43.671659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549d6d3-5930-4236-bae8-a9ed08b66b5a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:43.764531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cec85d2-1ee8-4a55-ae96-cec85cd3038a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:43.796652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:43.811806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6e65-cbfb-413d-ae29-27dfc326b610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:43.812436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6e65-cbfb-413d-ae29-27dfc326b610 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:43.812718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6e65-cbfb-413d-ae29-27dfc326b610 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:43.812989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6e65-cbfb-413d-ae29-27dfc326b610 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:43.822416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e6e65-cbfb-413d-ae29-27dfc326b610 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:43.824717 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e6e65-cbfb-413d-ae29-27dfc326b610 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:43.828788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-101e6e65-cbfb-413d-ae29-27dfc326b610 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:43.841762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:43.848448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:43.849594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 385/758] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Aug 9 16:32:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:43.852910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6e65-cbfb-413d-ae29-27dfc326b610 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:43.869994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549d6d3-5930-4236-bae8-a9ed08b66b5a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:43.929615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b228c3-0cf1-4da6-868b-92e1b1d089f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:43.930229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b228c3-0cf1-4da6-868b-92e1b1d089f8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:43.930500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b228c3-0cf1-4da6-868b-92e1b1d089f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:43.930760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b228c3-0cf1-4da6-868b-92e1b1d089f8 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:43.931445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b228c3-0cf1-4da6-868b-92e1b1d089f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:43.931736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b228c3-0cf1-4da6-868b-92e1b1d089f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:43.933140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b228c3-0cf1-4da6-868b-92e1b1d089f8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:43.947254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 374/759] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:43.962966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b228c3-0cf1-4da6-868b-92e1b1d089f8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:43.997374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6e65-cbfb-413d-ae29-27dfc326b610 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:44.032872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21696db3-c1be-42ec-869d-b1eabf0e9c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:44.033391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21696db3-c1be-42ec-869d-b1eabf0e9c3e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:44.033659 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21696db3-c1be-42ec-869d-b1eabf0e9c3e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:44.033919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21696db3-c1be-42ec-869d-b1eabf0e9c3e None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:44.034657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21696db3-c1be-42ec-869d-b1eabf0e9c3e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:44.034952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21696db3-c1be-42ec-869d-b1eabf0e9c3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:44.036366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21696db3-c1be-42ec-869d-b1eabf0e9c3e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:44.058489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 386/760] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Aug 9 16:32:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:44.060306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21696db3-c1be-42ec-869d-b1eabf0e9c3e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:44.110895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:44.166267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21696db3-c1be-42ec-869d-b1eabf0e9c3e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:44.202143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694f31c-fbc3-4a44-be96-8515a15cf7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:44.203023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694f31c-fbc3-4a44-be96-8515a15cf7e2 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:44.203436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694f31c-fbc3-4a44-be96-8515a15cf7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:44.203861 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0694f31c-fbc3-4a44-be96-8515a15cf7e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:44.204396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b228c3-0cf1-4da6-868b-92e1b1d089f8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:44.217358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694f31c-fbc3-4a44-be96-8515a15cf7e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:44.219141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0694f31c-fbc3-4a44-be96-8515a15cf7e2 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-0694f31c-fbc3-4a44-be96-8515a15cf7e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:44.234455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:44.235428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:44.236295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:44.236295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:44.236295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:44.242048 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 387/761] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:44.244502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694f31c-fbc3-4a44-be96-8515a15cf7e2 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:44.294857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd23d1-c28c-4a76-8acc-86f5f9c42c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:44.295479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd23d1-c28c-4a76-8acc-86f5f9c42c1a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:44.295768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd23d1-c28c-4a76-8acc-86f5f9c42c1a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:44.296314 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd23d1-c28c-4a76-8acc-86f5f9c42c1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:44.296767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd23d1-c28c-4a76-8acc-86f5f9c42c1a None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=35371b488c474b7f851af16bdad34f69) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:44.297073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd23d1-c28c-4a76-8acc-86f5f9c42c1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:44.298644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd23d1-c28c-4a76-8acc-86f5f9c42c1a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:44.306768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:44.312174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 375/762] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:43 2022] GET /identity/v3/role_assignments?user.id=35371b488c474b7f851af16bdad34f69&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 259 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:44.352340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:44.356953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0694f31c-fbc3-4a44-be96-8515a15cf7e2 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:44.390463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f5e55-9b89-438e-8ed8-4d766a68580e None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:44.391406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f5e55-9b89-438e-8ed8-4d766a68580e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:44.391873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f5e55-9b89-438e-8ed8-4d766a68580e None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:44.394217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 376/763] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:44.403429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5b403-510d-4d20-9d18-b359f6549136 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:44.403728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5b403-510d-4d20-9d18-b359f6549136 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:44.403976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5b403-510d-4d20-9d18-b359f6549136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:44.408929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195bd57-8202-4f61-bc9b-0c76fcaed804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:44.409549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195bd57-8202-4f61-bc9b-0c76fcaed804 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:44.409828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195bd57-8202-4f61-bc9b-0c76fcaed804 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:44.410142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195bd57-8202-4f61-bc9b-0c76fcaed804 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:44.410812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195bd57-8202-4f61-bc9b-0c76fcaed804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:44.411118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195bd57-8202-4f61-bc9b-0c76fcaed804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:44.412528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195bd57-8202-4f61-bc9b-0c76fcaed804 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:44.442157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-32e5b403-510d-4d20-9d18-b359f6549136 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:44.446327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-32e5b403-510d-4d20-9d18-b359f6549136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:44.449902 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 388/764] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:44.450911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195bd57-8202-4f61-bc9b-0c76fcaed804 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:44.539165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 377/765] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:32:44.578335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5b403-510d-4d20-9d18-b359f6549136 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:44.631018 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195bd57-8202-4f61-bc9b-0c76fcaed804 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:44.675324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592c0c8-cbbc-48e4-8600-4681dbb9691c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:44.675948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592c0c8-cbbc-48e4-8600-4681dbb9691c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:44.676220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592c0c8-cbbc-48e4-8600-4681dbb9691c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:44.676480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592c0c8-cbbc-48e4-8600-4681dbb9691c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:44.691124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592c0c8-cbbc-48e4-8600-4681dbb9691c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:44.692624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592c0c8-cbbc-48e4-8600-4681dbb9691c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-9592c0c8-cbbc-48e4-8600-4681dbb9691c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:44.708349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:44.709316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:44.710326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:44.710326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:44.710326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:44.710326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:44.710326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:44.710326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:44.710326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:44.710326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:44.712774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 389/766] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:44.720165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592c0c8-cbbc-48e4-8600-4681dbb9691c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:44.757709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:44.775080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5b403-510d-4d20-9d18-b359f6549136 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:44.810322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:44.815993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f542f2-b963-4c22-8e82-78cbc68ffd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:44.815993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f542f2-b963-4c22-8e82-78cbc68ffd3c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:44.815993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f542f2-b963-4c22-8e82-78cbc68ffd3c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:44.816496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f542f2-b963-4c22-8e82-78cbc68ffd3c None admin] PATH_INFO: `/v3/services/network` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:44.816668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f542f2-b963-4c22-8e82-78cbc68ffd3c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:44.818208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f542f2-b963-4c22-8e82-78cbc68ffd3c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-17f542f2-b963-4c22-8e82-78cbc68ffd3c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:44.836600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:44.838982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 16:32:44.841002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:44.845987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 378/767] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:44 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:44.846647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f542f2-b963-4c22-8e82-78cbc68ffd3c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:44.920254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592c0c8-cbbc-48e4-8600-4681dbb9691c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:45.012228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f542f2-b963-4c22-8e82-78cbc68ffd3c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:45.027596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ac44e-1b88-4ec0-8bfa-89badab7f24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:45.028198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ac44e-1b88-4ec0-8bfa-89badab7f24e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.028477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ac44e-1b88-4ec0-8bfa-89badab7f24e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.028735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ac44e-1b88-4ec0-8bfa-89badab7f24e None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.029447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ac44e-1b88-4ec0-8bfa-89badab7f24e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:45.029807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ac44e-1b88-4ec0-8bfa-89badab7f24e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:45.031245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ac44e-1b88-4ec0-8bfa-89badab7f24e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:45.048895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30acbf-5422-405c-9ea9-c528cbc9f1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:45.049597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30acbf-5422-405c-9ea9-c528cbc9f1a4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.049873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30acbf-5422-405c-9ea9-c528cbc9f1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.050163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30acbf-5422-405c-9ea9-c528cbc9f1a4 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.050796 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30acbf-5422-405c-9ea9-c528cbc9f1a4 None admin] RBAC: Adding query filter params (name=network) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:45.051090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30acbf-5422-405c-9ea9-c528cbc9f1a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:45.052426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30acbf-5422-405c-9ea9-c528cbc9f1a4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:45.058021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 390/768] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:45.063009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 379/769] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:32:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:45.064025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ac44e-1b88-4ec0-8bfa-89badab7f24e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:45.068302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30acbf-5422-405c-9ea9-c528cbc9f1a4 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:45.133597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:45.177335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30acbf-5422-405c-9ea9-c528cbc9f1a4 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:45.213868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa1cd2-5f08-430b-b3b9-c8816935f4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:45.214491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa1cd2-5f08-430b-b3b9-c8816935f4ce None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.214763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa1cd2-5f08-430b-b3b9-c8816935f4ce None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.215025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa1cd2-5f08-430b-b3b9-c8816935f4ce None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.215643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa1cd2-5f08-430b-b3b9-c8816935f4ce None admin] RBAC: Adding query filter params (type=network) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:45.215935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa1cd2-5f08-430b-b3b9-c8816935f4ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:45.217536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa1cd2-5f08-430b-b3b9-c8816935f4ce None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:45.223838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 380/770] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:32:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:45.229547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa1cd2-5f08-430b-b3b9-c8816935f4ce None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:45.253427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:45.312063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ac44e-1b88-4ec0-8bfa-89badab7f24e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:45.333663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa1cd2-5f08-430b-b3b9-c8816935f4ce None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:45.360128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:45.369364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336849d-925f-4a5f-95a1-776540d2d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:45.369981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336849d-925f-4a5f-95a1-776540d2d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.370272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336849d-925f-4a5f-95a1-776540d2d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.370535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336849d-925f-4a5f-95a1-776540d2d9d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.371227 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336849d-925f-4a5f-95a1-776540d2d9d0 None admin] RBAC: Adding query filter params (interface=public, service_id=584bfdb9f946416f8ea2558d37fe1d7d, region_id=RegionOne) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:45.371524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336849d-925f-4a5f-95a1-776540d2d9d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:45.373004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336849d-925f-4a5f-95a1-776540d2d9d0 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:45.381026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 381/771] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Aug 9 16:32:45 2022] GET /identity/v3/endpoints?service_id=584bfdb9f946416f8ea2558d37fe1d7d&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:45.386635 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336849d-925f-4a5f-95a1-776540d2d9d0 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:45.400764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f265f-d930-4e76-89ff-c9182ed3ee8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:45.401429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f265f-d930-4e76-89ff-c9182ed3ee8e None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.401701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f265f-d930-4e76-89ff-c9182ed3ee8e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.401984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f265f-d930-4e76-89ff-c9182ed3ee8e None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/5d17aa4b23e04290b6c8b1924b0dcf32/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.403890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f265f-d930-4e76-89ff-c9182ed3ee8e None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=5d17aa4b23e04290b6c8b1924b0dcf32, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:45.405378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f265f-d930-4e76-89ff-c9182ed3ee8e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:45.489055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 391/772] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:32:45 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/5d17aa4b23e04290b6c8b1924b0dcf32/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:32:45.491518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336849d-925f-4a5f-95a1-776540d2d9d0 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:45.532299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1246911d-d2a4-4a8c-b183-91bb495b8c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:45.532837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d5bba-9d92-4f14-ab2b-e849ca94ac9e None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.532837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d5bba-9d92-4f14-ab2b-e849ca94ac9e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.533142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246911d-d2a4-4a8c-b183-91bb495b8c11 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.533287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d5bba-9d92-4f14-ab2b-e849ca94ac9e None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.533462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246911d-d2a4-4a8c-b183-91bb495b8c11 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.533462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1246911d-d2a4-4a8c-b183-91bb495b8c11 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.533462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246911d-d2a4-4a8c-b183-91bb495b8c11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:45.533883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 392/773] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:45.534063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1246911d-d2a4-4a8c-b183-91bb495b8c11 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:45.534220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22953f22-35ab-4b91-9e13-b15ceedaa24b None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.537993 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22953f22-35ab-4b91-9e13-b15ceedaa24b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.537993 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22953f22-35ab-4b91-9e13-b15ceedaa24b None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.540641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 382/774] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Aug 9 16:32:45 2022] GET /identity/v3/services => generated 772 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:45.541445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 393/775] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:45.541913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfb872-157e-443d-926d-7f0c4438b768 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.542212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfb872-157e-443d-926d-7f0c4438b768 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.542480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfb872-157e-443d-926d-7f0c4438b768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.547132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b623a-d438-49fb-87c7-542c9f05ba25 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.547301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b623a-d438-49fb-87c7-542c9f05ba25 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.547808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b623a-d438-49fb-87c7-542c9f05ba25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.583385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2cdfb872-157e-443d-926d-7f0c4438b768 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:45.584643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2cdfb872-157e-443d-926d-7f0c4438b768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:45.593045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-4e7b623a-d438-49fb-87c7-542c9f05ba25 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:45.594286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4e7b623a-d438-49fb-87c7-542c9f05ba25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:45.693238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:45.722652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 394/776] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:32:45.724884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 383/777] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:32:45.747605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b623a-d438-49fb-87c7-542c9f05ba25 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:45.748354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfb872-157e-443d-926d-7f0c4438b768 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:45.846351 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:45.865385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfb872-157e-443d-926d-7f0c4438b768 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:45.891540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b623a-d438-49fb-87c7-542c9f05ba25 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:45.905769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91eccc-d0d8-4e76-b424-6ff6ce3f69c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:45.906510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91eccc-d0d8-4e76-b424-6ff6ce3f69c8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.906968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91eccc-d0d8-4e76-b424-6ff6ce3f69c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.907105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91eccc-d0d8-4e76-b424-6ff6ce3f69c8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.907661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91eccc-d0d8-4e76-b424-6ff6ce3f69c8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:45.909066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91eccc-d0d8-4e76-b424-6ff6ce3f69c8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-fd91eccc-d0d8-4e76-b424-6ff6ce3f69c8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:45.920177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:45.921252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:45.922097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 384/778] 10.222.0.79 () {56 vars in 1129 bytes} [Tue Aug 9 16:32:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:45.930223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91eccc-d0d8-4e76-b424-6ff6ce3f69c8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:45.948036 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79af79-a478-46da-ae86-c3362ef6cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:45.948585 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79af79-a478-46da-ae86-c3362ef6cd31 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:45.948855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79af79-a478-46da-ae86-c3362ef6cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:45.949122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79af79-a478-46da-ae86-c3362ef6cd31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:45.968652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c79af79-a478-46da-ae86-c3362ef6cd31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-4c79af79-a478-46da-ae86-c3362ef6cd31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:45.973436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:45.974781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:45.975601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 395/779] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:45.980552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79af79-a478-46da-ae86-c3362ef6cd31 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:46.025136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:46.106624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79af79-a478-46da-ae86-c3362ef6cd31 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:46.149900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:46.154133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb37cbb-0e1a-482a-becc-1bd69df90982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:46.155164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb37cbb-0e1a-482a-becc-1bd69df90982 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:46.155771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb37cbb-0e1a-482a-becc-1bd69df90982 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:46.156557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb37cbb-0e1a-482a-becc-1bd69df90982 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:46.157453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb37cbb-0e1a-482a-becc-1bd69df90982 None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:46.157923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb37cbb-0e1a-482a-becc-1bd69df90982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:46.159681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb37cbb-0e1a-482a-becc-1bd69df90982 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:46.162747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91eccc-d0d8-4e76-b424-6ff6ce3f69c8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:46.171562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 396/780] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:46.178915 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb37cbb-0e1a-482a-becc-1bd69df90982 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:46.211349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12c7e8-9bde-411b-9698-f18b4120a8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:46.211349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12c7e8-9bde-411b-9698-f18b4120a8e8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:46.211349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12c7e8-9bde-411b-9698-f18b4120a8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:46.211349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf12c7e8-9bde-411b-9698-f18b4120a8e8 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:46.211755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12c7e8-9bde-411b-9698-f18b4120a8e8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:46.211755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12c7e8-9bde-411b-9698-f18b4120a8e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:46.211755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf12c7e8-9bde-411b-9698-f18b4120a8e8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:46.214413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 385/781] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Aug 9 16:32:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:46.219877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12c7e8-9bde-411b-9698-f18b4120a8e8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:46.328645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb37cbb-0e1a-482a-becc-1bd69df90982 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:46.400231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0add58-c841-4109-a446-0bcd1f08caa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:46.400855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0add58-c841-4109-a446-0bcd1f08caa0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:46.401129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0add58-c841-4109-a446-0bcd1f08caa0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:46.401401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0add58-c841-4109-a446-0bcd1f08caa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:46.413767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf12c7e8-9bde-411b-9698-f18b4120a8e8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:46.420164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0add58-c841-4109-a446-0bcd1f08caa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:46.421635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0add58-c841-4109-a446-0bcd1f08caa0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:46.434811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-4c0add58-c841-4109-a446-0bcd1f08caa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:46.453543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:46.454567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:46.455433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:46.455433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:46.455433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:46.455433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 397/782] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:46.467444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0add58-c841-4109-a446-0bcd1f08caa0 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:46.472299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8888d2-d91a-4cc7-bd83-6fd186b8889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:46.472904 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8888d2-d91a-4cc7-bd83-6fd186b8889c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:46.473181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8888d2-d91a-4cc7-bd83-6fd186b8889c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:46.473440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8888d2-d91a-4cc7-bd83-6fd186b8889c None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:46.474791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8888d2-d91a-4cc7-bd83-6fd186b8889c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:46.475093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8888d2-d91a-4cc7-bd83-6fd186b8889c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:46.476475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8888d2-d91a-4cc7-bd83-6fd186b8889c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:46.489515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 386/783] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Aug 9 16:32:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:46.496789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8888d2-d91a-4cc7-bd83-6fd186b8889c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:46.553293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:46.641162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0add58-c841-4109-a446-0bcd1f08caa0 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:46.664834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8888d2-d91a-4cc7-bd83-6fd186b8889c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:46.678429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb447f6a-c554-45fc-ab50-6dec6dc1cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:46.679161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb447f6a-c554-45fc-ab50-6dec6dc1cb6f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:46.679439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb447f6a-c554-45fc-ab50-6dec6dc1cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:46.679708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb447f6a-c554-45fc-ab50-6dec6dc1cb6f None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:46.680377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb447f6a-c554-45fc-ab50-6dec6dc1cb6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:46.680676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb447f6a-c554-45fc-ab50-6dec6dc1cb6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:46.682249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb447f6a-c554-45fc-ab50-6dec6dc1cb6f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:46.699465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b701a61-629e-4555-9624-1e0ad9419341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:46.699921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 398/784] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:46.700343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b701a61-629e-4555-9624-1e0ad9419341 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:46.700615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b701a61-629e-4555-9624-1e0ad9419341 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:46.700878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b701a61-629e-4555-9624-1e0ad9419341 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:46.701230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e0d05-a4a2-4271-ab8a-7d960b7c7eb9 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:46.701517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e0d05-a4a2-4271-ab8a-7d960b7c7eb9 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:46.701617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b701a61-629e-4555-9624-1e0ad9419341 None admin] RBAC: Adding query filter params (interface=public, service_id=ea89e0c483c848e6a3177a4453cf9ddc, region_id=RegionOne) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:46.701786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e0d05-a4a2-4271-ab8a-7d960b7c7eb9 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:46.701896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b701a61-629e-4555-9624-1e0ad9419341 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:46.703286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 399/785] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:46.703396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b701a61-629e-4555-9624-1e0ad9419341 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:46.703986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e0d05-a4a2-4271-ab8a-7d960b7c7eb9 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:46.708888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 387/786] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Aug 9 16:32:46 2022] GET /identity/v3/endpoints?service_id=ea89e0c483c848e6a3177a4453cf9ddc&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:46.710053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0b062-aa80-498d-a618-30ebd80f087c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:46.710894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0b062-aa80-498d-a618-30ebd80f087c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:46.711163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0b062-aa80-498d-a618-30ebd80f087c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:46.729610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-fef0b062-aa80-498d-a618-30ebd80f087c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:46.732664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fef0b062-aa80-498d-a618-30ebd80f087c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:46.761938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:46.809976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 388/787] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:32:46.809976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0b062-aa80-498d-a618-30ebd80f087c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:46.840165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e0d05-a4a2-4271-ab8a-7d960b7c7eb9 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:46.842416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:46.884109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14331f19-505e-4de8-b05c-a87a858f979e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:46.885006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14331f19-505e-4de8-b05c-a87a858f979e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:46.885444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14331f19-505e-4de8-b05c-a87a858f979e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:46.886209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14331f19-505e-4de8-b05c-a87a858f979e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:46.894838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14331f19-505e-4de8-b05c-a87a858f979e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:46.896485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14331f19-505e-4de8-b05c-a87a858f979e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-14331f19-505e-4de8-b05c-a87a858f979e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:46.909825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:46.910817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:46.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:46.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:46.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:46.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:46.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:46.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:46.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:46.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:46.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:32:46.911765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:46.912285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0b062-aa80-498d-a618-30ebd80f087c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:46.919175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 400/788] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Aug 9 16:32:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:46.920594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14331f19-505e-4de8-b05c-a87a858f979e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:46.949793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd19da-474f-4987-89d0-5549cd13ca3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:46.950609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd19da-474f-4987-89d0-5549cd13ca3b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:46.950723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd19da-474f-4987-89d0-5549cd13ca3b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:46.950955 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd19da-474f-4987-89d0-5549cd13ca3b None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:46.951668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd19da-474f-4987-89d0-5549cd13ca3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:46.953019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd19da-474f-4987-89d0-5549cd13ca3b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:46.958660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 389/789] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Aug 9 16:32:46 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:46.959527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd19da-474f-4987-89d0-5549cd13ca3b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:47.008107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:47.043259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14331f19-505e-4de8-b05c-a87a858f979e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:47.088288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2915e-779c-4877-9b68-9deea736d6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:47.089390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2915e-779c-4877-9b68-9deea736d6a3 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:47.089989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2915e-779c-4877-9b68-9deea736d6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:47.090768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2915e-779c-4877-9b68-9deea736d6a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:47.118926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2915e-779c-4877-9b68-9deea736d6a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:47.130496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd19da-474f-4987-89d0-5549cd13ca3b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-caf2915e-779c-4877-9b68-9deea736d6a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:47.134788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:47.135706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:47.136529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:47.139259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 401/790] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:47.146793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2915e-779c-4877-9b68-9deea736d6a3 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:47.176439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:47.177384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf9895-5c2b-40dc-8a75-eecc7188305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:47.178218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf9895-5c2b-40dc-8a75-eecc7188305b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:47.179560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf9895-5c2b-40dc-8a75-eecc7188305b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:47.180120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf9895-5c2b-40dc-8a75-eecc7188305b None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:47.181167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf9895-5c2b-40dc-8a75-eecc7188305b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:47.181789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf9895-5c2b-40dc-8a75-eecc7188305b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:47.184006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf9895-5c2b-40dc-8a75-eecc7188305b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:47.209298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 390/791] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Aug 9 16:32:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:47.215562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf9895-5c2b-40dc-8a75-eecc7188305b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:47.281550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2915e-779c-4877-9b68-9deea736d6a3 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:47.318373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4c7d2-2094-4fca-919e-5782e53c80a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:47.319393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4c7d2-2094-4fca-919e-5782e53c80a7 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:47.320411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4c7d2-2094-4fca-919e-5782e53c80a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:47.320880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4c7d2-2094-4fca-919e-5782e53c80a7 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:47.321746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4c7d2-2094-4fca-919e-5782e53c80a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:47.322277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4c7d2-2094-4fca-919e-5782e53c80a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:47.323999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4c7d2-2094-4fca-919e-5782e53c80a7 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:47.333400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 402/792] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:47.347280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4c7d2-2094-4fca-919e-5782e53c80a7 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:47.347692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:47.362928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf9895-5c2b-40dc-8a75-eecc7188305b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:47.405783 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021a2d9-a624-42e3-92fc-d01b7356af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:47.407779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021a2d9-a624-42e3-92fc-d01b7356af26 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:47.409380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021a2d9-a624-42e3-92fc-d01b7356af26 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:47.409875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021a2d9-a624-42e3-92fc-d01b7356af26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:47.433555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021a2d9-a624-42e3-92fc-d01b7356af26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:47.436968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021a2d9-a624-42e3-92fc-d01b7356af26 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-5021a2d9-a624-42e3-92fc-d01b7356af26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:47.465294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:47.466832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:47.467957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:47.467957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:47.467957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:47.476517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 391/793] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:47.483268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021a2d9-a624-42e3-92fc-d01b7356af26 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:47.559777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:47.598222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4c7d2-2094-4fca-919e-5782e53c80a7 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:47.652583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3b687-b0fe-49a9-b67d-e0784d6ea75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:47.653207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b687-b0fe-49a9-b67d-e0784d6ea75d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:47.653489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b687-b0fe-49a9-b67d-e0784d6ea75d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:47.653760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3b687-b0fe-49a9-b67d-e0784d6ea75d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:47.660356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021a2d9-a624-42e3-92fc-d01b7356af26 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:47.665367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3b687-b0fe-49a9-b67d-e0784d6ea75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:47.666861 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3b687-b0fe-49a9-b67d-e0784d6ea75d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-d1e3b687-b0fe-49a9-b67d-e0784d6ea75d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:47.681642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:47.682693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:47.683574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:47.683574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:47.683574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:47.683574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 403/794] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:47.695013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3b687-b0fe-49a9-b67d-e0784d6ea75d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:47.727105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5940cf-5f14-4651-908d-255e1c77b3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:47.727722 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5940cf-5f14-4651-908d-255e1c77b3db None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:47.727999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5940cf-5f14-4651-908d-255e1c77b3db None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:47.728267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5940cf-5f14-4651-908d-255e1c77b3db None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:47.728914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5940cf-5f14-4651-908d-255e1c77b3db None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:47.729213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5940cf-5f14-4651-908d-255e1c77b3db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:47.730604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5940cf-5f14-4651-908d-255e1c77b3db None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:47.756526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 392/795] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:47.758301 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:47.761485 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5940cf-5f14-4651-908d-255e1c77b3db None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:47.850047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3b687-b0fe-49a9-b67d-e0784d6ea75d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:47.894355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e179c-a0d5-4e7a-888f-01aed7ae273b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:47.894955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e179c-a0d5-4e7a-888f-01aed7ae273b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:47.895230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e179c-a0d5-4e7a-888f-01aed7ae273b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:47.895498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e179c-a0d5-4e7a-888f-01aed7ae273b None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:47.896138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e179c-a0d5-4e7a-888f-01aed7ae273b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:47.896437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e179c-a0d5-4e7a-888f-01aed7ae273b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:47.897837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e179c-a0d5-4e7a-888f-01aed7ae273b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:47.905292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:47.930477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 404/796] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:47.937306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e179c-a0d5-4e7a-888f-01aed7ae273b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:47.940102 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5940cf-5f14-4651-908d-255e1c77b3db None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:47.980893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be9d4b-e553-44c9-b5f4-c01af10393c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:47.981861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be9d4b-e553-44c9-b5f4-c01af10393c7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:47.982583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be9d4b-e553-44c9-b5f4-c01af10393c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:47.983896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be9d4b-e553-44c9-b5f4-c01af10393c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:47.992142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be9d4b-e553-44c9-b5f4-c01af10393c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:47.994525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be9d4b-e553-44c9-b5f4-c01af10393c7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-c8be9d4b-e553-44c9-b5f4-c01af10393c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:48.010707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:48.011631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:48.012523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:48.012523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:48.012523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:48.012523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:48.012523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:48.012523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:48.012523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:48.012523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:48.015330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 393/797] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:48.020621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9983bf-3bb9-4940-9e3a-9ea9144561ec None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.020621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9983bf-3bb9-4940-9e3a-9ea9144561ec None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.020621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9983bf-3bb9-4940-9e3a-9ea9144561ec None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.022022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 394/798] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:48.023651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9983bf-3bb9-4940-9e3a-9ea9144561ec None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:48.065783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e179c-a0d5-4e7a-888f-01aed7ae273b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:48.093122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:48.105079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05c02d2-2dca-4607-903a-956a03b0ac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:48.106112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c02d2-2dca-4607-903a-956a03b0ac79 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.106592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c02d2-2dca-4607-903a-956a03b0ac79 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.107160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05c02d2-2dca-4607-903a-956a03b0ac79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.115330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c02d2-2dca-4607-903a-956a03b0ac79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:48.117586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05c02d2-2dca-4607-903a-956a03b0ac79 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-b05c02d2-2dca-4607-903a-956a03b0ac79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:48.131704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:48.132706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:48.133627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:48.133627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:48.133627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:48.133627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:48.133627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:48.133627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:48.133627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:48.133627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:48.133627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:48.133627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:48.136966 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 405/799] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:48.138448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779a5b9-53c8-4d63-bb21-d9e3cbdcaf9d None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.138952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779a5b9-53c8-4d63-bb21-d9e3cbdcaf9d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.139434 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d779a5b9-53c8-4d63-bb21-d9e3cbdcaf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.139538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9983bf-3bb9-4940-9e3a-9ea9144561ec None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:48.160574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d779a5b9-53c8-4d63-bb21-d9e3cbdcaf9d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:48.162452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d779a5b9-53c8-4d63-bb21-d9e3cbdcaf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:48.175802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a4325-6e43-4908-887b-8ad2ffc8f27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:48.176406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a4325-6e43-4908-887b-8ad2ffc8f27a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.176686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a4325-6e43-4908-887b-8ad2ffc8f27a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.176957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a4325-6e43-4908-887b-8ad2ffc8f27a None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.177641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a4325-6e43-4908-887b-8ad2ffc8f27a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:48.177944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a4325-6e43-4908-887b-8ad2ffc8f27a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:48.179989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a4325-6e43-4908-887b-8ad2ffc8f27a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:48.202002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 395/800] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:48.202002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a4325-6e43-4908-887b-8ad2ffc8f27a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:48.227973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:48.246586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 406/801] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:32:48.248320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e4d5-ae67-4a33-a63c-dbad2e643cd6 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.248679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e4d5-ae67-4a33-a63c-dbad2e643cd6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.248956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e4d5-ae67-4a33-a63c-dbad2e643cd6 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.250509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 407/802] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:48.251623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6e4d5-ae67-4a33-a63c-dbad2e643cd6 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:48.313982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a4325-6e43-4908-887b-8ad2ffc8f27a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:48.337819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:48.351985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158f1ac-d8ae-4a36-872e-d610f4551698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:48.352586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158f1ac-d8ae-4a36-872e-d610f4551698 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.352861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158f1ac-d8ae-4a36-872e-d610f4551698 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.353125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158f1ac-d8ae-4a36-872e-d610f4551698 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.353843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158f1ac-d8ae-4a36-872e-d610f4551698 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:48.354259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158f1ac-d8ae-4a36-872e-d610f4551698 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:48.355622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158f1ac-d8ae-4a36-872e-d610f4551698 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:48.379749 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 396/803] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Aug 9 16:32:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:48.379947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6e4d5-ae67-4a33-a63c-dbad2e643cd6 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:48.380364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a1816-cfc3-4706-832c-72268034aa1a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.380364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a1816-cfc3-4706-832c-72268034aa1a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.380364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a1816-cfc3-4706-832c-72268034aa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.398667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d67a1816-cfc3-4706-832c-72268034aa1a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:48.399863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d67a1816-cfc3-4706-832c-72268034aa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:48.415374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df373e7c-bf5c-4866-a29c-5bf3cce2f509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:48.416024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df373e7c-bf5c-4866-a29c-5bf3cce2f509 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.416298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df373e7c-bf5c-4866-a29c-5bf3cce2f509 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.416568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df373e7c-bf5c-4866-a29c-5bf3cce2f509 None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/a010ea3529644fca951a7eef28dbe97e/roles/844d711adf9b4c1f8471a01e6031e1e6` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.433231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df373e7c-bf5c-4866-a29c-5bf3cce2f509 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=a010ea3529644fca951a7eef28dbe97e, role_id=844d711adf9b4c1f8471a01e6031e1e6)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:48.435553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df373e7c-bf5c-4866-a29c-5bf3cce2f509 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:48.477537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 397/804] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:32:48.478414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a1816-cfc3-4706-832c-72268034aa1a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:48.507467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 408/805] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:32:48 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/a010ea3529644fca951a7eef28dbe97e/roles/844d711adf9b4c1f8471a01e6031e1e6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:48.508817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df373e7c-bf5c-4866-a29c-5bf3cce2f509 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:48.564736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:48.631862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a1816-cfc3-4706-832c-72268034aa1a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:48.647036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:48.662713 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df373e7c-bf5c-4866-a29c-5bf3cce2f509 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:48.676184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab9376-8e11-4475-802c-10ad6e0323eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:48.676881 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab9376-8e11-4475-802c-10ad6e0323eb None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.677096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab9376-8e11-4475-802c-10ad6e0323eb None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.677350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab9376-8e11-4475-802c-10ad6e0323eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.692794 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab9376-8e11-4475-802c-10ad6e0323eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-7aab9376-8e11-4475-802c-10ad6e0323eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:48.699507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:48.700427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:48.701422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 398/806] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:48.702184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab9376-8e11-4475-802c-10ad6e0323eb None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:48.711356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3102e-efe9-4000-9e19-4bc12af55aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:48.712102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3102e-efe9-4000-9e19-4bc12af55aef None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.712389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3102e-efe9-4000-9e19-4bc12af55aef None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.712673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee3102e-efe9-4000-9e19-4bc12af55aef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.733442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3102e-efe9-4000-9e19-4bc12af55aef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:48.735225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee3102e-efe9-4000-9e19-4bc12af55aef None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-eee3102e-efe9-4000-9e19-4bc12af55aef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:48.755967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:48.756947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:48.758258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:48.758258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:48.758258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:48.758258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 409/807] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:48.758503 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3102e-efe9-4000-9e19-4bc12af55aef None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:48.827758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab9376-8e11-4475-802c-10ad6e0323eb None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:48.866702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:48.869059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e0199-8aef-4ec9-8fec-e4e21d8fcfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:48.869731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e0199-8aef-4ec9-8fec-e4e21d8fcfc6 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.870059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e0199-8aef-4ec9-8fec-e4e21d8fcfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.870377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e0199-8aef-4ec9-8fec-e4e21d8fcfc6 None admin] PATH_INFO: `/v3/services/network` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.870982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e0199-8aef-4ec9-8fec-e4e21d8fcfc6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:48.876589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e0199-8aef-4ec9-8fec-e4e21d8fcfc6 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:48.880181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee3102e-efe9-4000-9e19-4bc12af55aef None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-f07e0199-8aef-4ec9-8fec-e4e21d8fcfc6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:48.882045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:48.883017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 09 16:32:48.884498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:48.888111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 399/808] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:48 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:48.889200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e0199-8aef-4ec9-8fec-e4e21d8fcfc6 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:48.920375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ba53f-545f-4f04-bb22-b09259fe8f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:48.921315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ba53f-545f-4f04-bb22-b09259fe8f0f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:48.921802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ba53f-545f-4f04-bb22-b09259fe8f0f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:48.922303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ba53f-545f-4f04-bb22-b09259fe8f0f None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:48.923308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ba53f-545f-4f04-bb22-b09259fe8f0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:48.923885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ba53f-545f-4f04-bb22-b09259fe8f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:48.925432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ba53f-545f-4f04-bb22-b09259fe8f0f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:48.926898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:48.935112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 410/809] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:48.941072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ba53f-545f-4f04-bb22-b09259fe8f0f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:49.006197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e0199-8aef-4ec9-8fec-e4e21d8fcfc6 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:49.045073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67f183-0a38-49e9-b99d-94d058bd151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:49.045681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67f183-0a38-49e9-b99d-94d058bd151a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.045976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67f183-0a38-49e9-b99d-94d058bd151a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.046259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67f183-0a38-49e9-b99d-94d058bd151a None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.047391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67f183-0a38-49e9-b99d-94d058bd151a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:49.048124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67f183-0a38-49e9-b99d-94d058bd151a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:49.051075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67f183-0a38-49e9-b99d-94d058bd151a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:49.052741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ba53f-545f-4f04-bb22-b09259fe8f0f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:49.070640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 400/810] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:49.071951 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67f183-0a38-49e9-b99d-94d058bd151a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:49.095947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5ffe1-0ac2-494e-8228-8dcf8a7a738c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:49.096745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5ffe1-0ac2-494e-8228-8dcf8a7a738c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.097124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5ffe1-0ac2-494e-8228-8dcf8a7a738c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.097458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5ffe1-0ac2-494e-8228-8dcf8a7a738c None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.098263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5ffe1-0ac2-494e-8228-8dcf8a7a738c None admin] RBAC: Adding query filter params (name=network) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:49.098630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5ffe1-0ac2-494e-8228-8dcf8a7a738c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:49.100258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5ffe1-0ac2-494e-8228-8dcf8a7a738c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:49.109871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 411/811] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:32:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:49.111030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5ffe1-0ac2-494e-8228-8dcf8a7a738c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:49.154452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:49.183221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:49.191763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67f183-0a38-49e9-b99d-94d058bd151a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:49.218953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5ffe1-0ac2-494e-8228-8dcf8a7a738c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:49.227984 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432f728-f48d-4c5a-9100-e576e08f30cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:49.228608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432f728-f48d-4c5a-9100-e576e08f30cd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.228879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432f728-f48d-4c5a-9100-e576e08f30cd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.229142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432f728-f48d-4c5a-9100-e576e08f30cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.244730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432f728-f48d-4c5a-9100-e576e08f30cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:49.246527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432f728-f48d-4c5a-9100-e576e08f30cd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-c432f728-f48d-4c5a-9100-e576e08f30cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:49.264045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:49.265197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:49.266319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:49.266319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:49.266319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:49.268824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 401/812] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:49.269965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432f728-f48d-4c5a-9100-e576e08f30cd None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:49.277255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aba307-a5f0-4da5-bbdf-0314b6277670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:49.278158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aba307-a5f0-4da5-bbdf-0314b6277670 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.278578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aba307-a5f0-4da5-bbdf-0314b6277670 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.279000 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aba307-a5f0-4da5-bbdf-0314b6277670 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.288054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aba307-a5f0-4da5-bbdf-0314b6277670 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:49.290642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aba307-a5f0-4da5-bbdf-0314b6277670 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-27aba307-a5f0-4da5-bbdf-0314b6277670 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:49.306709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:49.307724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:49.308628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:49.308628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:49.308628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:49.308628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:49.308628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:49.308628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:49.308628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:49.308628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:49.315862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 412/813] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:49.315862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aba307-a5f0-4da5-bbdf-0314b6277670 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:49.391112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432f728-f48d-4c5a-9100-e576e08f30cd None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:49.396499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:49.425534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aba307-a5f0-4da5-bbdf-0314b6277670 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:49.431135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab11b12-7099-41a9-bea2-01f66257d427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:49.431736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab11b12-7099-41a9-bea2-01f66257d427 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.432013 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab11b12-7099-41a9-bea2-01f66257d427 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.432281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab11b12-7099-41a9-bea2-01f66257d427 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.432910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab11b12-7099-41a9-bea2-01f66257d427 None admin] RBAC: Adding query filter params (type=network) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:49.433210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab11b12-7099-41a9-bea2-01f66257d427 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:49.434586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab11b12-7099-41a9-bea2-01f66257d427 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:49.440600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 402/814] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:32:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:49.441974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a8b13-b37d-4d20-bfa3-41a27132b142 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.442703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a8b13-b37d-4d20-bfa3-41a27132b142 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.442961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a8b13-b37d-4d20-bfa3-41a27132b142 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.444286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 403/815] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:49.445641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a8b13-b37d-4d20-bfa3-41a27132b142 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:49.459644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:49.474114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317601d7-1661-415b-bcc2-73ccd6adcdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:49.476382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317601d7-1661-415b-bcc2-73ccd6adcdc9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.476382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317601d7-1661-415b-bcc2-73ccd6adcdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.476382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317601d7-1661-415b-bcc2-73ccd6adcdc9 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.476382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317601d7-1661-415b-bcc2-73ccd6adcdc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:49.476382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317601d7-1661-415b-bcc2-73ccd6adcdc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:49.477664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317601d7-1661-415b-bcc2-73ccd6adcdc9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:49.496251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 413/816] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:49.496251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317601d7-1661-415b-bcc2-73ccd6adcdc9 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:49.583814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a8b13-b37d-4d20-bfa3-41a27132b142 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:49.611622 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317601d7-1661-415b-bcc2-73ccd6adcdc9 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:49.624114 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8979d-313d-4e91-92f1-995b654075ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:49.624930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8979d-313d-4e91-92f1-995b654075ab None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.625372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8979d-313d-4e91-92f1-995b654075ab None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.625806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8979d-313d-4e91-92f1-995b654075ab None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.626524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8979d-313d-4e91-92f1-995b654075ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:49.627035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8979d-313d-4e91-92f1-995b654075ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:49.628226 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8979d-313d-4e91-92f1-995b654075ab None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:49.646786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 404/817] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:49.646786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f0492-f321-4d32-ad0d-7d2f24fe70dd None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.646786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f0492-f321-4d32-ad0d-7d2f24fe70dd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.646786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f0492-f321-4d32-ad0d-7d2f24fe70dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.653226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4da74c-c62e-41e8-b2de-5233ef8a324e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:49.653824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4da74c-c62e-41e8-b2de-5233ef8a324e None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.654154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4da74c-c62e-41e8-b2de-5233ef8a324e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.654457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4da74c-c62e-41e8-b2de-5233ef8a324e None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.655079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4da74c-c62e-41e8-b2de-5233ef8a324e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:49.656456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4da74c-c62e-41e8-b2de-5233ef8a324e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:49.661746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:49.662595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-842f0492-f321-4d32-ad0d-7d2f24fe70dd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:49.663982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-842f0492-f321-4d32-ad0d-7d2f24fe70dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:49.728836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:49.734477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 405/818] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:32:49.735393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f0492-f321-4d32-ad0d-7d2f24fe70dd None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:49.753994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 414/819] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:32:49 2022] POST /identity/v3/endpoints => generated 337 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:32:49.753994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4da74c-c62e-41e8-b2de-5233ef8a324e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:49.868743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f0492-f321-4d32-ad0d-7d2f24fe70dd None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:49.908604 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4da74c-c62e-41e8-b2de-5233ef8a324e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:49.913008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2533be-ff62-4cf7-9279-aa5514e0e93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:49.914214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2533be-ff62-4cf7-9279-aa5514e0e93d None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.914482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2533be-ff62-4cf7-9279-aa5514e0e93d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.914741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2533be-ff62-4cf7-9279-aa5514e0e93d None admin] PATH_INFO: `/v3/users/nova` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.922494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2533be-ff62-4cf7-9279-aa5514e0e93d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:49.924097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2533be-ff62-4cf7-9279-aa5514e0e93d None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-4f2533be-ff62-4cf7-9279-aa5514e0e93d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:49.937195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:49.938283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 406/820] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Aug 9 16:32:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:49.939209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2533be-ff62-4cf7-9279-aa5514e0e93d None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:49.955690 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdacab-a406-403f-9881-87d0080619c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:49.956353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdacab-a406-403f-9881-87d0080619c9 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:49.956677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdacab-a406-403f-9881-87d0080619c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:49.956951 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdacab-a406-403f-9881-87d0080619c9 None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/35371b488c474b7f851af16bdad34f69/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:49.958984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdacab-a406-403f-9881-87d0080619c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=35371b488c474b7f851af16bdad34f69, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:49.960447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdacab-a406-403f-9881-87d0080619c9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:50.004508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:50.048142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2533be-ff62-4cf7-9279-aa5514e0e93d None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:50.059567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 415/821] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:32:49 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/35371b488c474b7f851af16bdad34f69/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:32:50.060648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdacab-a406-403f-9881-87d0080619c9 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:50.098509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc61ec5-89a7-45bf-b2df-a1c88dc9a048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:50.099440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc61ec5-89a7-45bf-b2df-a1c88dc9a048 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:50.099854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc61ec5-89a7-45bf-b2df-a1c88dc9a048 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:50.100429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc61ec5-89a7-45bf-b2df-a1c88dc9a048 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:50.102348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc61ec5-89a7-45bf-b2df-a1c88dc9a048 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:50.103761 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc61ec5-89a7-45bf-b2df-a1c88dc9a048 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-9cc61ec5-89a7-45bf-b2df-a1c88dc9a048 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:50.121312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:50.122401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 09 16:32:50.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:50.123961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:50.126125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 407/822] 10.222.0.79 () {56 vars in 1129 bytes} [Tue Aug 9 16:32:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:50.129915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc61ec5-89a7-45bf-b2df-a1c88dc9a048 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:50.192219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdacab-a406-403f-9881-87d0080619c9 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:50.240124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e826a2d-5bc8-4aff-9a0f-33c4189b5133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:50.241083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e826a2d-5bc8-4aff-9a0f-33c4189b5133 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:50.241603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e826a2d-5bc8-4aff-9a0f-33c4189b5133 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:50.242285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e826a2d-5bc8-4aff-9a0f-33c4189b5133 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:50.243670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e826a2d-5bc8-4aff-9a0f-33c4189b5133 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:50.244412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e826a2d-5bc8-4aff-9a0f-33c4189b5133 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:50.246146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e826a2d-5bc8-4aff-9a0f-33c4189b5133 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:50.262295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc61ec5-89a7-45bf-b2df-a1c88dc9a048 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:50.265173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 416/823] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Aug 9 16:32:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:50.270654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e826a2d-5bc8-4aff-9a0f-33c4189b5133 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:50.294391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:50.318758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88eabc-c041-4fa8-a967-9def530ff113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:50.319616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88eabc-c041-4fa8-a967-9def530ff113 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:50.320039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88eabc-c041-4fa8-a967-9def530ff113 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:50.320480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88eabc-c041-4fa8-a967-9def530ff113 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:50.321827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88eabc-c041-4fa8-a967-9def530ff113 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:50.322332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88eabc-c041-4fa8-a967-9def530ff113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:50.324521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88eabc-c041-4fa8-a967-9def530ff113 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:50.332530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 408/824] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Aug 9 16:32:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:50.338271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88eabc-c041-4fa8-a967-9def530ff113 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:50.384592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e826a2d-5bc8-4aff-9a0f-33c4189b5133 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:50.387797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:50.420988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041e7fb-23fd-403d-80f2-cab6057d9aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:50.422776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041e7fb-23fd-403d-80f2-cab6057d9aaf None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:50.422776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041e7fb-23fd-403d-80f2-cab6057d9aaf None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:50.423245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041e7fb-23fd-403d-80f2-cab6057d9aaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:50.474962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88eabc-c041-4fa8-a967-9def530ff113 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:50.513710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906aa59-c1d4-49a0-995e-9b7fa3f6b554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:50.514597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906aa59-c1d4-49a0-995e-9b7fa3f6b554 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:50.515052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906aa59-c1d4-49a0-995e-9b7fa3f6b554 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:50.515468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906aa59-c1d4-49a0-995e-9b7fa3f6b554 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:50.516246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906aa59-c1d4-49a0-995e-9b7fa3f6b554 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:50.516981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906aa59-c1d4-49a0-995e-9b7fa3f6b554 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:50.517671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041e7fb-23fd-403d-80f2-cab6057d9aaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:50.518690 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d906aa59-c1d4-49a0-995e-9b7fa3f6b554 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:50.519396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041e7fb-23fd-403d-80f2-cab6057d9aaf None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:50.526768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 409/825] 10.222.0.79 () {56 vars in 1124 bytes} [Tue Aug 9 16:32:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:50.537583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906aa59-c1d4-49a0-995e-9b7fa3f6b554 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:50.610333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:50.648692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 417/826] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:32:50.650809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041e7fb-23fd-403d-80f2-cab6057d9aaf None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:50.654297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:50.667698 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906aa59-c1d4-49a0-995e-9b7fa3f6b554 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:50.704566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d47f53-a55d-4f07-80c1-2bd3e4115da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:50.705167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47f53-a55d-4f07-80c1-2bd3e4115da2 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:50.705444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47f53-a55d-4f07-80c1-2bd3e4115da2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:50.705711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d47f53-a55d-4f07-80c1-2bd3e4115da2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:50.706522 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d47f53-a55d-4f07-80c1-2bd3e4115da2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:50.707891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d47f53-a55d-4f07-80c1-2bd3e4115da2 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:50.754657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041e7fb-23fd-403d-80f2-cab6057d9aaf None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:50.790496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 410/827] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:32:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:32:50.792240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a848e-7ace-4172-9e23-01510f4c6017 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:50.792531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a848e-7ace-4172-9e23-01510f4c6017 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:50.792809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a848e-7ace-4172-9e23-01510f4c6017 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:50.794363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 411/828] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:50.796588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd1e8a-5fbb-4414-bdf5-47c9b725f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:50.797281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd1e8a-5fbb-4414-bdf5-47c9b725f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:50.797624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd1e8a-5fbb-4414-bdf5-47c9b725f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:50.797972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd1e8a-5fbb-4414-bdf5-47c9b725f4c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:50.799982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9be65-d996-49bf-bcdc-f20ae6937986 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:50.800509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9be65-d996-49bf-bcdc-f20ae6937986 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:50.803268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9be65-d996-49bf-bcdc-f20ae6937986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:50.815968 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd1e8a-5fbb-4414-bdf5-47c9b725f4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:50.817923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bd1e8a-5fbb-4414-bdf5-47c9b725f4c5 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:50.823953 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b7d9be65-d996-49bf-bcdc-f20ae6937986 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:50.825342 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b7d9be65-d996-49bf-bcdc-f20ae6937986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-13bd1e8a-5fbb-4414-bdf5-47c9b725f4c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:50.837394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:50.838657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:50.839777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:50.839777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:50.839777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:50.846586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 418/829] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:50.848333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd1e8a-5fbb-4414-bdf5-47c9b725f4c5 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:50.912648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 412/830] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:32:50.913322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:50.937212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9be65-d996-49bf-bcdc-f20ae6937986 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:50.970497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bd1e8a-5fbb-4414-bdf5-47c9b725f4c5 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:50.972927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:51.020967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace0a47-8f4a-4239-8750-7f486d9ab1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:51.021897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0a47-8f4a-4239-8750-7f486d9ab1f0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:51.022192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0a47-8f4a-4239-8750-7f486d9ab1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:51.022461 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace0a47-8f4a-4239-8750-7f486d9ab1f0 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:51.023105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace0a47-8f4a-4239-8750-7f486d9ab1f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:51.023389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace0a47-8f4a-4239-8750-7f486d9ab1f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:51.024745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace0a47-8f4a-4239-8750-7f486d9ab1f0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:51.037911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 419/831] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:51.044010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace0a47-8f4a-4239-8750-7f486d9ab1f0 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:51.057703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9be65-d996-49bf-bcdc-f20ae6937986 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:51.094142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85274-dc74-4ab1-b520-ec1e0cc64441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:51.095035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85274-dc74-4ab1-b520-ec1e0cc64441 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:51.095478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85274-dc74-4ab1-b520-ec1e0cc64441 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:51.095896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85274-dc74-4ab1-b520-ec1e0cc64441 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:51.107292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb85274-dc74-4ab1-b520-ec1e0cc64441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-feb85274-dc74-4ab1-b520-ec1e0cc64441 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:51.112610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:51.113718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:32:51.114790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:51.117577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 413/832] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:32:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:51.126197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85274-dc74-4ab1-b520-ec1e0cc64441 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:51.158190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace0a47-8f4a-4239-8750-7f486d9ab1f0 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:51.173903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:51.194634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba127c-07da-4899-b785-67bbe073bf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:51.195876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba127c-07da-4899-b785-67bbe073bf85 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:51.196218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba127c-07da-4899-b785-67bbe073bf85 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:51.196569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba127c-07da-4899-b785-67bbe073bf85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:51.203912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba127c-07da-4899-b785-67bbe073bf85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:51.205642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba127c-07da-4899-b785-67bbe073bf85 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-c5ba127c-07da-4899-b785-67bbe073bf85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:51.218252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:51.219232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:51.220105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:51.220105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:51.220105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:51.220105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:51.220105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:51.220105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:51.220105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:51.220105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:51.223505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 420/833] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:51.226106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba127c-07da-4899-b785-67bbe073bf85 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:51.244200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:51.253615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85274-dc74-4ab1-b520-ec1e0cc64441 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:51.291517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc3523-8fe8-4525-9e21-790eb255532f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:51.292296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc3523-8fe8-4525-9e21-790eb255532f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:51.292724 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc3523-8fe8-4525-9e21-790eb255532f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:51.293090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc3523-8fe8-4525-9e21-790eb255532f None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:51.293850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc3523-8fe8-4525-9e21-790eb255532f None admin] RBAC: Adding query filter params (name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:51.294337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc3523-8fe8-4525-9e21-790eb255532f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:51.295981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc3523-8fe8-4525-9e21-790eb255532f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:51.309435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 414/834] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:32:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:51.311015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc3523-8fe8-4525-9e21-790eb255532f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:51.344492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba127c-07da-4899-b785-67bbe073bf85 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:51.389527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4104a-9096-4ae4-8ec2-7d96f5e49671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:51.390183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4104a-9096-4ae4-8ec2-7d96f5e49671 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:51.390466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4104a-9096-4ae4-8ec2-7d96f5e49671 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:51.390737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4104a-9096-4ae4-8ec2-7d96f5e49671 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:51.391596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4104a-9096-4ae4-8ec2-7d96f5e49671 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:51.391906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4104a-9096-4ae4-8ec2-7d96f5e49671 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:51.394158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4104a-9096-4ae4-8ec2-7d96f5e49671 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:51.413997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 421/835] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:51.415690 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4104a-9096-4ae4-8ec2-7d96f5e49671 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:51.430778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc3523-8fe8-4525-9e21-790eb255532f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:51.456083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:51.468148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5dc0ac-092d-4c1e-bd29-1757004144ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:51.468869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dc0ac-092d-4c1e-bd29-1757004144ec None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:51.469237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dc0ac-092d-4c1e-bd29-1757004144ec None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:51.469596 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dc0ac-092d-4c1e-bd29-1757004144ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:51.480528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5dc0ac-092d-4c1e-bd29-1757004144ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:51.482179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5dc0ac-092d-4c1e-bd29-1757004144ec None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-0a5dc0ac-092d-4c1e-bd29-1757004144ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:51.496802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:51.498033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:51.499577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:51.499577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:51.499577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:51.504970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 415/836] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:51.505972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5dc0ac-092d-4c1e-bd29-1757004144ec None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:51.520551 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:51.552361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4104a-9096-4ae4-8ec2-7d96f5e49671 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:51.589970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c90ff6-06dc-4b78-aaca-588e15f2b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:51.590595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c90ff6-06dc-4b78-aaca-588e15f2b29e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:51.590867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c90ff6-06dc-4b78-aaca-588e15f2b29e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:51.591133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c90ff6-06dc-4b78-aaca-588e15f2b29e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:51.592157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c90ff6-06dc-4b78-aaca-588e15f2b29e None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=5d17aa4b23e04290b6c8b1924b0dcf32) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:51.592394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c90ff6-06dc-4b78-aaca-588e15f2b29e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:51.593923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c90ff6-06dc-4b78-aaca-588e15f2b29e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:51.606035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 422/837] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:51 2022] GET /identity/v3/role_assignments?user.id=5d17aa4b23e04290b6c8b1924b0dcf32&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:51.643068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5dc0ac-092d-4c1e-bd29-1757004144ec None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:51.683487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10768ddb-a29d-4be2-aba5-ddf46f49e87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:51.684502 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10768ddb-a29d-4be2-aba5-ddf46f49e87a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:51.684998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10768ddb-a29d-4be2-aba5-ddf46f49e87a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:51.685474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10768ddb-a29d-4be2-aba5-ddf46f49e87a None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:51.686375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10768ddb-a29d-4be2-aba5-ddf46f49e87a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:51.686884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10768ddb-a29d-4be2-aba5-ddf46f49e87a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:51.688629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10768ddb-a29d-4be2-aba5-ddf46f49e87a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:51.703573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 416/838] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:51.709993 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c90ff6-06dc-4b78-aaca-588e15f2b29e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:51.808908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:51.815190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c90ff6-06dc-4b78-aaca-588e15f2b29e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:51.852371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10567ad-5779-477b-b872-25c0e497bc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:51.853369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10567ad-5779-477b-b872-25c0e497bc84 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:51.853851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10567ad-5779-477b-b872-25c0e497bc84 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:51.854352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10567ad-5779-477b-b872-25c0e497bc84 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:51.862771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10567ad-5779-477b-b872-25c0e497bc84 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:51.864511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10567ad-5779-477b-b872-25c0e497bc84 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-b10567ad-5779-477b-b872-25c0e497bc84 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:51.876623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:51.877799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:51.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:51.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:51.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:51.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:51.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:51.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:51.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:51.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:51.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:32:51.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:51.881115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 423/839] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Aug 9 16:32:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:51.890802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10768ddb-a29d-4be2-aba5-ddf46f49e87a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:51.964383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:52.012077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10768ddb-a29d-4be2-aba5-ddf46f49e87a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:52.057205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b540c46-dab0-4368-94ad-a482cf6ae9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:52.057816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b540c46-dab0-4368-94ad-a482cf6ae9c2 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:52.058127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b540c46-dab0-4368-94ad-a482cf6ae9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:52.058397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b540c46-dab0-4368-94ad-a482cf6ae9c2 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:52.059280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b540c46-dab0-4368-94ad-a482cf6ae9c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:52.059525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b540c46-dab0-4368-94ad-a482cf6ae9c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:52.061160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b540c46-dab0-4368-94ad-a482cf6ae9c2 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:52.081717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 417/840] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Aug 9 16:32:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:52.087549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10567ad-5779-477b-b872-25c0e497bc84 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:52.169799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabf8fa-bb7a-41e3-884f-df48ff15a445 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:52.170355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabf8fa-bb7a-41e3-884f-df48ff15a445 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:52.170637 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabf8fa-bb7a-41e3-884f-df48ff15a445 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:52.172257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 418/841] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:52.178333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108d67b-d9d0-47de-9dd2-514b7758fadf None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:52.179835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108d67b-d9d0-47de-9dd2-514b7758fadf None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:52.180489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108d67b-d9d0-47de-9dd2-514b7758fadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:52.199189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-3108d67b-d9d0-47de-9dd2-514b7758fadf None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:52.199977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10567ad-5779-477b-b872-25c0e497bc84 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:52.200949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3108d67b-d9d0-47de-9dd2-514b7758fadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:52.244350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72e686-54d0-4cea-8395-64c6e57defdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:52.244959 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72e686-54d0-4cea-8395-64c6e57defdc None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:52.245264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72e686-54d0-4cea-8395-64c6e57defdc None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:52.245544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72e686-54d0-4cea-8395-64c6e57defdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:52.291169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 419/842] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:32:52.303378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:52.317536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108d67b-d9d0-47de-9dd2-514b7758fadf None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:52.353071 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72e686-54d0-4cea-8395-64c6e57defdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:52.354888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72e686-54d0-4cea-8395-64c6e57defdc None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:52.382983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:52.449739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108d67b-d9d0-47de-9dd2-514b7758fadf None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:52.485172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 424/843] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:32:52.490956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72e686-54d0-4cea-8395-64c6e57defdc None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:52.493373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7929045-f681-4253-8f3b-cc2f22d28b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:52.494106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7929045-f681-4253-8f3b-cc2f22d28b03 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:52.494360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7929045-f681-4253-8f3b-cc2f22d28b03 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:52.494619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7929045-f681-4253-8f3b-cc2f22d28b03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:52.508647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7929045-f681-4253-8f3b-cc2f22d28b03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-f7929045-f681-4253-8f3b-cc2f22d28b03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:52.515554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:52.516564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:32:52.517393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:52.521697 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 420/844] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:52.522892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7929045-f681-4253-8f3b-cc2f22d28b03 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:52.572094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:52.622655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72e686-54d0-4cea-8395-64c6e57defdc None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:52.639643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7929045-f681-4253-8f3b-cc2f22d28b03 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:52.655964 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:52.661857 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1434516-2c9c-4a6a-b5ff-64ace1fdda8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:52.662818 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1434516-2c9c-4a6a-b5ff-64ace1fdda8b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:52.663277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1434516-2c9c-4a6a-b5ff-64ace1fdda8b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:52.663749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1434516-2c9c-4a6a-b5ff-64ace1fdda8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:52.675227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1434516-2c9c-4a6a-b5ff-64ace1fdda8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:52.677424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1434516-2c9c-4a6a-b5ff-64ace1fdda8b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:52.680381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8764d9-1ea8-407c-813a-15c956b60b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:52.680928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8764d9-1ea8-407c-813a-15c956b60b3e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:52.682934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8764d9-1ea8-407c-813a-15c956b60b3e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:52.682934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8764d9-1ea8-407c-813a-15c956b60b3e None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:52.682934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8764d9-1ea8-407c-813a-15c956b60b3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:52.682934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8764d9-1ea8-407c-813a-15c956b60b3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:52.684534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8764d9-1ea8-407c-813a-15c956b60b3e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-d1434516-2c9c-4a6a-b5ff-64ace1fdda8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:52.696374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:52.697374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:52.698281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:52.698281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:52.698281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:52.699223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 421/845] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:32:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:52.700852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8764d9-1ea8-407c-813a-15c956b60b3e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:52.701492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 425/846] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:52.708049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1434516-2c9c-4a6a-b5ff-64ace1fdda8b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:52.831077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1434516-2c9c-4a6a-b5ff-64ace1fdda8b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:52.834826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8764d9-1ea8-407c-813a-15c956b60b3e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:52.840961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:52.868772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70302531-76ed-45fd-9800-cf41d23752bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:52.869375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70302531-76ed-45fd-9800-cf41d23752bd None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:52.869626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70302531-76ed-45fd-9800-cf41d23752bd None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:52.869898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70302531-76ed-45fd-9800-cf41d23752bd None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:52.870712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70302531-76ed-45fd-9800-cf41d23752bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:52.871570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70302531-76ed-45fd-9800-cf41d23752bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:52.873635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70302531-76ed-45fd-9800-cf41d23752bd None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:52.879553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6568e6-ff33-4040-8b38-34174b293ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:52.880499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6568e6-ff33-4040-8b38-34174b293ffd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:52.880915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6568e6-ff33-4040-8b38-34174b293ffd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:52.881320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6568e6-ff33-4040-8b38-34174b293ffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:52.892582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 426/847] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:52.893739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70302531-76ed-45fd-9800-cf41d23752bd None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:52.897738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6568e6-ff33-4040-8b38-34174b293ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:52.899924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6568e6-ff33-4040-8b38-34174b293ffd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-cb6568e6-ff33-4040-8b38-34174b293ffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:52.916061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:52.917240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:52.918334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:52.918334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:52.918334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:52.925621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 422/848] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:52.926804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6568e6-ff33-4040-8b38-34174b293ffd None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:52.959363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:53.017606 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70302531-76ed-45fd-9800-cf41d23752bd None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:53.042255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6568e6-ff33-4040-8b38-34174b293ffd None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:53.069311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27736f54-8e80-491e-9936-fad5025c3215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:53.070480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27736f54-8e80-491e-9936-fad5025c3215 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.070750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27736f54-8e80-491e-9936-fad5025c3215 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.071011 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27736f54-8e80-491e-9936-fad5025c3215 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:53.078587 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27736f54-8e80-491e-9936-fad5025c3215 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:53.080026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27736f54-8e80-491e-9936-fad5025c3215 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-27736f54-8e80-491e-9936-fad5025c3215 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:53.094774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:53.096672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:53.098927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:53.098927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:53.098927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:53.098927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:53.098927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:53.098927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:53.098927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:53.098927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:53.100714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 427/849] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:53.102329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ceb92-1a22-4204-9fac-cc048eace502 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.102503 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ceb92-1a22-4204-9fac-cc048eace502 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.102774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ceb92-1a22-4204-9fac-cc048eace502 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:53.104228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad02ae2-5c99-4ed8-8ecf-7a74f0074048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:53.104403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 428/850] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:53.104680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ceb92-1a22-4204-9fac-cc048eace502 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:53.111057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad02ae2-5c99-4ed8-8ecf-7a74f0074048 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.111492 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad02ae2-5c99-4ed8-8ecf-7a74f0074048 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.111911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad02ae2-5c99-4ed8-8ecf-7a74f0074048 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:53.112933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad02ae2-5c99-4ed8-8ecf-7a74f0074048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:53.113399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad02ae2-5c99-4ed8-8ecf-7a74f0074048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:53.115641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad02ae2-5c99-4ed8-8ecf-7a74f0074048 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:53.139053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 423/851] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:53.141010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fb527-50f2-4f05-864e-007f9a878764 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.141289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fb527-50f2-4f05-864e-007f9a878764 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.141556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994fb527-50f2-4f05-864e-007f9a878764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:53.159379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:53.162989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-994fb527-50f2-4f05-864e-007f9a878764 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:53.164264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-994fb527-50f2-4f05-864e-007f9a878764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:53.256680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ceb92-1a22-4204-9fac-cc048eace502 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:53.277993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 424/852] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:32:53.277993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fb527-50f2-4f05-864e-007f9a878764 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:53.307717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:53.318531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3f056-72cf-4ddb-96c6-8cff1a66da61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:53.319430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3f056-72cf-4ddb-96c6-8cff1a66da61 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.320396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3f056-72cf-4ddb-96c6-8cff1a66da61 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.320812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3f056-72cf-4ddb-96c6-8cff1a66da61 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:53.321738 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3f056-72cf-4ddb-96c6-8cff1a66da61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:53.322220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3f056-72cf-4ddb-96c6-8cff1a66da61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:53.323836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3f056-72cf-4ddb-96c6-8cff1a66da61 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:53.336584 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 429/853] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:53.337670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3f056-72cf-4ddb-96c6-8cff1a66da61 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:53.414272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994fb527-50f2-4f05-864e-007f9a878764 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:53.450312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:53.459408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841a75-c6ac-403a-a3f9-23e5859e836d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:53.472097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841a75-c6ac-403a-a3f9-23e5859e836d None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.472097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841a75-c6ac-403a-a3f9-23e5859e836d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.472097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47841a75-c6ac-403a-a3f9-23e5859e836d None admin] PATH_INFO: `/v3/users/swift` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:53.472344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3f056-72cf-4ddb-96c6-8cff1a66da61 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:53.472419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47841a75-c6ac-403a-a3f9-23e5859e836d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:53.472419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47841a75-c6ac-403a-a3f9-23e5859e836d None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-47841a75-c6ac-403a-a3f9-23e5859e836d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:53.485221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:53.486548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 425/854] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:32:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:53.493035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841a75-c6ac-403a-a3f9-23e5859e836d None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:53.499170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a13db-0275-44f9-94c2-324f3c24f67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:53.499804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a13db-0275-44f9-94c2-324f3c24f67e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.500083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a13db-0275-44f9-94c2-324f3c24f67e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.500339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a13db-0275-44f9-94c2-324f3c24f67e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:53.500923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a13db-0275-44f9-94c2-324f3c24f67e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:53.502430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a13db-0275-44f9-94c2-324f3c24f67e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-0e5a13db-0275-44f9-94c2-324f3c24f67e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:53.512593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:53.513761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:53.515004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 430/855] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:53.516250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a13db-0275-44f9-94c2-324f3c24f67e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:53.590806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:53.615516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47841a75-c6ac-403a-a3f9-23e5859e836d None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:53.633419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a13db-0275-44f9-94c2-324f3c24f67e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:53.652581 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9c74a-2a32-437b-81de-138895f05f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:53.653180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9c74a-2a32-437b-81de-138895f05f75 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.653452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9c74a-2a32-437b-81de-138895f05f75 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.653856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9c74a-2a32-437b-81de-138895f05f75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:53.654608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9c74a-2a32-437b-81de-138895f05f75 None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=a010ea3529644fca951a7eef28dbe97e) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:53.654906 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9c74a-2a32-437b-81de-138895f05f75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:53.656318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9c74a-2a32-437b-81de-138895f05f75 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:53.662422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 426/856] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:53 2022] GET /identity/v3/role_assignments?user.id=a010ea3529644fca951a7eef28dbe97e&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:53.663238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9c74a-2a32-437b-81de-138895f05f75 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:53.683779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa388b1-e236-42df-8da9-0b1f2c61938e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:53.683779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa388b1-e236-42df-8da9-0b1f2c61938e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.683779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa388b1-e236-42df-8da9-0b1f2c61938e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.684183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa388b1-e236-42df-8da9-0b1f2c61938e None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:53.684877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa388b1-e236-42df-8da9-0b1f2c61938e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:53.685188 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa388b1-e236-42df-8da9-0b1f2c61938e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:53.686849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa388b1-e236-42df-8da9-0b1f2c61938e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:53.718890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 431/857] 10.222.0.79 () {56 vars in 1135 bytes} [Tue Aug 9 16:32:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:53.720017 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa388b1-e236-42df-8da9-0b1f2c61938e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:53.734569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:53.810683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9c74a-2a32-437b-81de-138895f05f75 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:53.851925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea104a5-ed49-4fea-98cf-1967ac57eed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:53.852165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea104a5-ed49-4fea-98cf-1967ac57eed9 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.852528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea104a5-ed49-4fea-98cf-1967ac57eed9 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.852893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea104a5-ed49-4fea-98cf-1967ac57eed9 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:53.853667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea104a5-ed49-4fea-98cf-1967ac57eed9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:53.854091 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea104a5-ed49-4fea-98cf-1967ac57eed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:53.854970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa388b1-e236-42df-8da9-0b1f2c61938e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:53.855681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea104a5-ed49-4fea-98cf-1967ac57eed9 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:53.862618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 427/858] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:53.870713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea104a5-ed49-4fea-98cf-1967ac57eed9 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:53.880988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:53.901252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c50227-1e91-46fc-9754-f5f27b9cfc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:53.901844 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c50227-1e91-46fc-9754-f5f27b9cfc73 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:53.902197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c50227-1e91-46fc-9754-f5f27b9cfc73 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:53.902465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c50227-1e91-46fc-9754-f5f27b9cfc73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:54.009580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c50227-1e91-46fc-9754-f5f27b9cfc73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:54.011072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c50227-1e91-46fc-9754-f5f27b9cfc73 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:54.016072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea104a5-ed49-4fea-98cf-1967ac57eed9 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:54.030863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:54.057576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a30fd-1ccb-42ca-9dc7-5a833b633934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:54.058606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a30fd-1ccb-42ca-9dc7-5a833b633934 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:54.059096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a30fd-1ccb-42ca-9dc7-5a833b633934 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:54.059663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a30fd-1ccb-42ca-9dc7-5a833b633934 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:54.060627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a30fd-1ccb-42ca-9dc7-5a833b633934 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:54.061154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a30fd-1ccb-42ca-9dc7-5a833b633934 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:54.063547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a30fd-1ccb-42ca-9dc7-5a833b633934 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:54.069754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 428/859] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:53 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:54.178467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 432/860] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:32:54.184766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a30fd-1ccb-42ca-9dc7-5a833b633934 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:54.225382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585cdd6-aa4c-425a-84ae-0cced9272e45 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:54.225734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585cdd6-aa4c-425a-84ae-0cced9272e45 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:54.225924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585cdd6-aa4c-425a-84ae-0cced9272e45 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:54.232207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 433/861] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:54.234645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1ec1-b6bf-426d-a741-25679e6e5b1c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:54.235212 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1ec1-b6bf-426d-a741-25679e6e5b1c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:54.235754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1ec1-b6bf-426d-a741-25679e6e5b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:54.261437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-e0fb1ec1-b6bf-426d-a741-25679e6e5b1c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:54.262592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e0fb1ec1-b6bf-426d-a741-25679e6e5b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:54.302611 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:54.328723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a30fd-1ccb-42ca-9dc7-5a833b633934 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:54.345811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 434/862] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:32:54.373811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97f7ba-c2e7-49ed-b677-e4ed4ac5a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:54.374815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97f7ba-c2e7-49ed-b677-e4ed4ac5a6da None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:54.375241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97f7ba-c2e7-49ed-b677-e4ed4ac5a6da None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:54.375657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97f7ba-c2e7-49ed-b677-e4ed4ac5a6da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:54.387130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1ec1-b6bf-426d-a741-25679e6e5b1c None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:54.390082 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97f7ba-c2e7-49ed-b677-e4ed4ac5a6da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:54.391571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97f7ba-c2e7-49ed-b677-e4ed4ac5a6da None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-bf97f7ba-c2e7-49ed-b677-e4ed4ac5a6da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:54.410549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:54.411577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:54.413101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:54.413101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:54.413101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:54.413101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 429/863] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:54.417708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97f7ba-c2e7-49ed-b677-e4ed4ac5a6da None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:54.449579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:54.517037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1ec1-b6bf-426d-a741-25679e6e5b1c None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:54.537217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97f7ba-c2e7-49ed-b677-e4ed4ac5a6da None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:54.562779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:54.568624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216a8de-23ff-445b-a87c-88cad80655eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:54.569480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216a8de-23ff-445b-a87c-88cad80655eb None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:54.570537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216a8de-23ff-445b-a87c-88cad80655eb None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:54.571127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216a8de-23ff-445b-a87c-88cad80655eb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:54.574701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216a8de-23ff-445b-a87c-88cad80655eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:54.574701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216a8de-23ff-445b-a87c-88cad80655eb None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:54.576391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836d760-caf3-4208-9c5d-3f576268a7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:54.577211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836d760-caf3-4208-9c5d-3f576268a7ea None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:54.577584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836d760-caf3-4208-9c5d-3f576268a7ea None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:54.577962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836d760-caf3-4208-9c5d-3f576268a7ea None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:54.578824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836d760-caf3-4208-9c5d-3f576268a7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:54.579219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836d760-caf3-4208-9c5d-3f576268a7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:54.580781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836d760-caf3-4208-9c5d-3f576268a7ea None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-b216a8de-23ff-445b-a87c-88cad80655eb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:54.586382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:54.587490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 16:32:54.588412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:54.592619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 435/864] 10.222.0.79 () {56 vars in 1132 bytes} [Tue Aug 9 16:32:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:54.601978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216a8de-23ff-445b-a87c-88cad80655eb None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:54.602189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 430/865] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:54.602396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836d760-caf3-4208-9c5d-3f576268a7ea None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:54.739399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836d760-caf3-4208-9c5d-3f576268a7ea None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:54.758943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216a8de-23ff-445b-a87c-88cad80655eb None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:54.762752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:54.784679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e687b2-90ee-4820-a8a4-b3bedda2b0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:54.785300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e687b2-90ee-4820-a8a4-b3bedda2b0e1 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:54.785572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e687b2-90ee-4820-a8a4-b3bedda2b0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:54.785840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e687b2-90ee-4820-a8a4-b3bedda2b0e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:54.795614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e687b2-90ee-4820-a8a4-b3bedda2b0e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:54.797616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e687b2-90ee-4820-a8a4-b3bedda2b0e1 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:54.803859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e9fd4-40f0-4d84-a02c-faa8a8417c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:54.805255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e9fd4-40f0-4d84-a02c-faa8a8417c59 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:54.805708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e9fd4-40f0-4d84-a02c-faa8a8417c59 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:54.806178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e9fd4-40f0-4d84-a02c-faa8a8417c59 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:54.807266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e9fd4-40f0-4d84-a02c-faa8a8417c59 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:54.807742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e9fd4-40f0-4d84-a02c-faa8a8417c59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:54.809285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62e9fd4-40f0-4d84-a02c-faa8a8417c59 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-27e687b2-90ee-4820-a8a4-b3bedda2b0e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:54.813117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:54.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:54.814962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:54.814962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:54.814962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:54.814962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:54.814962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:54.814962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:54.814962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:54.814962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:54.814962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 431/866] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:54.817262 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 436/867] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:32:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:54.823549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e687b2-90ee-4820-a8a4-b3bedda2b0e1 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:54.827129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e9fd4-40f0-4d84-a02c-faa8a8417c59 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:54.883662 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:54.948190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e687b2-90ee-4820-a8a4-b3bedda2b0e1 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:54.959871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e9fd4-40f0-4d84-a02c-faa8a8417c59 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:54.990070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273f2d6-4619-4a2c-ab85-bb777a2ff95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:54.990967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273f2d6-4619-4a2c-ab85-bb777a2ff95a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:54.991477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273f2d6-4619-4a2c-ab85-bb777a2ff95a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:54.992410 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273f2d6-4619-4a2c-ab85-bb777a2ff95a None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:54.993333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273f2d6-4619-4a2c-ab85-bb777a2ff95a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:54.993771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273f2d6-4619-4a2c-ab85-bb777a2ff95a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:54.995360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273f2d6-4619-4a2c-ab85-bb777a2ff95a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:55.003763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5297c2-787e-46bc-a933-3345140bb927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:55.004935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5297c2-787e-46bc-a933-3345140bb927 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:55.005217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5297c2-787e-46bc-a933-3345140bb927 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:55.005521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5297c2-787e-46bc-a933-3345140bb927 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:55.006160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5297c2-787e-46bc-a933-3345140bb927 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:55.006463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5297c2-787e-46bc-a933-3345140bb927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:55.009706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5297c2-787e-46bc-a933-3345140bb927 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:55.012250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 432/868] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:55.021992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 437/869] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:32:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:55.027848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273f2d6-4619-4a2c-ab85-bb777a2ff95a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:55.171855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273f2d6-4619-4a2c-ab85-bb777a2ff95a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:55.174062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:55.211956 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4644674-fea4-48c8-bb14-0ac1de210180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:55.213131 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4644674-fea4-48c8-bb14-0ac1de210180 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:55.213651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4644674-fea4-48c8-bb14-0ac1de210180 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:55.214228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4644674-fea4-48c8-bb14-0ac1de210180 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:55.215480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4644674-fea4-48c8-bb14-0ac1de210180 None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=35371b488c474b7f851af16bdad34f69) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:55.215791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4644674-fea4-48c8-bb14-0ac1de210180 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:55.217672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4644674-fea4-48c8-bb14-0ac1de210180 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:55.224858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 433/870] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:55 2022] GET /identity/v3/role_assignments?user.id=35371b488c474b7f851af16bdad34f69&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:56.073051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729cb17-e418-47f2-bdc4-fe0d7168db9e None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:56.073578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729cb17-e418-47f2-bdc4-fe0d7168db9e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:56.073578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729cb17-e418-47f2-bdc4-fe0d7168db9e None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:56.075166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 438/871] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:56.080650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6655be9-95ae-4494-9b06-1718c8a893f6 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:56.081066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6655be9-95ae-4494-9b06-1718c8a893f6 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:56.081207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6655be9-95ae-4494-9b06-1718c8a893f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:56.101176 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c6655be9-95ae-4494-9b06-1718c8a893f6 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:56.102379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c6655be9-95ae-4494-9b06-1718c8a893f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:56.173994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 434/872] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:32:56.197083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729cb17-e418-47f2-bdc4-fe0d7168db9e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:56.230802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:56.302180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729cb17-e418-47f2-bdc4-fe0d7168db9e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:56.337198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c4796-8cad-4a49-9e12-e9f47229bcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:56.337739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c4796-8cad-4a49-9e12-e9f47229bcae None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:56.338096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c4796-8cad-4a49-9e12-e9f47229bcae None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:56.338366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c4796-8cad-4a49-9e12-e9f47229bcae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:56.350577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c4796-8cad-4a49-9e12-e9f47229bcae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-736c4796-8cad-4a49-9e12-e9f47229bcae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:32:56.355214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:56.356114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:56.356937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 439/873] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Aug 9 16:32:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:56.362825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6655be9-95ae-4494-9b06-1718c8a893f6 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:56.466127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:56.472224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6655be9-95ae-4494-9b06-1718c8a893f6 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:56.501970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea348dcd-84f2-4940-bd01-55394adf38b7 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:56.502324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea348dcd-84f2-4940-bd01-55394adf38b7 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:56.502574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea348dcd-84f2-4940-bd01-55394adf38b7 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:56.504205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 440/874] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:56.511161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f810610-b98c-4f6a-b7ca-3a97c0d96623 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:56.511721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f810610-b98c-4f6a-b7ca-3a97c0d96623 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:56.512207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f810610-b98c-4f6a-b7ca-3a97c0d96623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:56.518110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e1905-e561-4f82-9a42-c9db716e0159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:56.518824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e1905-e561-4f82-9a42-c9db716e0159 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:56.519104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e1905-e561-4f82-9a42-c9db716e0159 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:56.519374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e1905-e561-4f82-9a42-c9db716e0159 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:56.520033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e1905-e561-4f82-9a42-c9db716e0159 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:56.521172 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e1905-e561-4f82-9a42-c9db716e0159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:56.521770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e1905-e561-4f82-9a42-c9db716e0159 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:56.533288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-7f810610-b98c-4f6a-b7ca-3a97c0d96623 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:56.534744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7f810610-b98c-4f6a-b7ca-3a97c0d96623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:56.537701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 435/875] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Aug 9 16:32:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:56.543804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e1905-e561-4f82-9a42-c9db716e0159 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:56.614149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 441/876] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:32:56.629444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f810610-b98c-4f6a-b7ca-3a97c0d96623 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:56.653655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e1905-e561-4f82-9a42-c9db716e0159 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:56.655114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:56.690616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ebae3-c5f3-4f3f-b5be-4c29d7d0b3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:56.691237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ebae3-c5f3-4f3f-b5be-4c29d7d0b3e4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:56.691551 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ebae3-c5f3-4f3f-b5be-4c29d7d0b3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:56.691765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ebae3-c5f3-4f3f-b5be-4c29d7d0b3e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:56.704841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ebae3-c5f3-4f3f-b5be-4c29d7d0b3e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:56.705806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ebae3-c5f3-4f3f-b5be-4c29d7d0b3e4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-d41ebae3-c5f3-4f3f-b5be-4c29d7d0b3e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:56.721931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:56.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:56.724260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:56.724260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:56.724260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:56.728037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 436/877] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:56.729153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ebae3-c5f3-4f3f-b5be-4c29d7d0b3e4 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:56.739602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f810610-b98c-4f6a-b7ca-3a97c0d96623 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:56.743250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:56.776385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cba74-3a4f-409b-8259-34007e63b7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:56.777376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cba74-3a4f-409b-8259-34007e63b7e4 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:56.777888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cba74-3a4f-409b-8259-34007e63b7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:56.778377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cba74-3a4f-409b-8259-34007e63b7e4 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:56.779202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cba74-3a4f-409b-8259-34007e63b7e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:56.780838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88cba74-3a4f-409b-8259-34007e63b7e4 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:56.838630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ebae3-c5f3-4f3f-b5be-4c29d7d0b3e4 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:56.866076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 442/878] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:32:56 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:32:56.889854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd42aba-e464-49dc-af55-8a94d32bf4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:56.890511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd42aba-e464-49dc-af55-8a94d32bf4a3 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:56.890785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd42aba-e464-49dc-af55-8a94d32bf4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:56.891097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd42aba-e464-49dc-af55-8a94d32bf4a3 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:56.891861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd42aba-e464-49dc-af55-8a94d32bf4a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:56.892203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd42aba-e464-49dc-af55-8a94d32bf4a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:56.893704 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd42aba-e464-49dc-af55-8a94d32bf4a3 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:56.908973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 437/879] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:56.914757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cba74-3a4f-409b-8259-34007e63b7e4 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:57.010778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:57.031511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88cba74-3a4f-409b-8259-34007e63b7e4 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:57.072843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff8306-66cb-418b-9ec3-277ab47f6766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:57.073460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff8306-66cb-418b-9ec3-277ab47f6766 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:57.074060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff8306-66cb-418b-9ec3-277ab47f6766 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:57.074060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff8306-66cb-418b-9ec3-277ab47f6766 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:57.082221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff8306-66cb-418b-9ec3-277ab47f6766 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:57.083703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff8306-66cb-418b-9ec3-277ab47f6766 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-c3ff8306-66cb-418b-9ec3-277ab47f6766 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:32:57.096316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:57.097312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:57.098922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 443/880] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Aug 9 16:32:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:57.103431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd42aba-e464-49dc-af55-8a94d32bf4a3 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:57.149631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580eff2-28cd-4a2c-9bcb-ffcee9f0b1c5 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:57.149822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580eff2-28cd-4a2c-9bcb-ffcee9f0b1c5 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:57.150174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580eff2-28cd-4a2c-9bcb-ffcee9f0b1c5 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:57.151733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 444/881] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:57.156969 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e546be-6882-46a5-b1ef-762f5e971a3c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:57.157255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e546be-6882-46a5-b1ef-762f5e971a3c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:57.157507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e546be-6882-46a5-b1ef-762f5e971a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:57.178897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:57.181012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c8e546be-6882-46a5-b1ef-762f5e971a3c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:57.183130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c8e546be-6882-46a5-b1ef-762f5e971a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:57.218608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd42aba-e464-49dc-af55-8a94d32bf4a3 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:57.259693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec10b6-e83e-40bf-909e-cdcb400bdcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:57.260634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec10b6-e83e-40bf-909e-cdcb400bdcd9 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:57.261077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec10b6-e83e-40bf-909e-cdcb400bdcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:57.261537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec10b6-e83e-40bf-909e-cdcb400bdcd9 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:57.263093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec10b6-e83e-40bf-909e-cdcb400bdcd9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:57.263650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec10b6-e83e-40bf-909e-cdcb400bdcd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:57.266192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec10b6-e83e-40bf-909e-cdcb400bdcd9 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:57.278038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 445/882] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:57 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:32:57.296170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 438/883] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Aug 9 16:32:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:57.302325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec10b6-e83e-40bf-909e-cdcb400bdcd9 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:57.302491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e546be-6882-46a5-b1ef-762f5e971a3c None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:57.389507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:57.424371 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec10b6-e83e-40bf-909e-cdcb400bdcd9 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:57.433386 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e546be-6882-46a5-b1ef-762f5e971a3c None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:57.450302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:57.462842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f4724-b571-4db1-b823-bded5f1579a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:57.463530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f4724-b571-4db1-b823-bded5f1579a4 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:57.463739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f4724-b571-4db1-b823-bded5f1579a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:57.463970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f4724-b571-4db1-b823-bded5f1579a4 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:57.464616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f4724-b571-4db1-b823-bded5f1579a4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:57.466044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f4724-b571-4db1-b823-bded5f1579a4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:57.477361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f976470-303e-440a-86c0-6abf57fbbc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:57.477993 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976470-303e-440a-86c0-6abf57fbbc5d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:57.478276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976470-303e-440a-86c0-6abf57fbbc5d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:57.478547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f976470-303e-440a-86c0-6abf57fbbc5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:57.560223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f976470-303e-440a-86c0-6abf57fbbc5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:57.563907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f976470-303e-440a-86c0-6abf57fbbc5d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:57.576453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 439/884] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:32:57 2022] POST /identity/v3/services => generated 269 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:32:57.577767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a88ac-ec56-41f4-a055-944bb18861e7 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:57.578082 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a88ac-ec56-41f4-a055-944bb18861e7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:57.589308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a88ac-ec56-41f4-a055-944bb18861e7 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:57.589308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 440/885] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:57.589308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abd178-b93d-4052-b8fe-1c1356317fbd None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:57.589308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abd178-b93d-4052-b8fe-1c1356317fbd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:57.589308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abd178-b93d-4052-b8fe-1c1356317fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:57.608702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b9abd178-b93d-4052-b8fe-1c1356317fbd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:57.611674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b9abd178-b93d-4052-b8fe-1c1356317fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:57.653698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:57.690971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 446/886] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:32:57 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:32:57.696541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f976470-303e-440a-86c0-6abf57fbbc5d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:57.725631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 441/887] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:32:57.749144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abd178-b93d-4052-b8fe-1c1356317fbd None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:57.820943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f976470-303e-440a-86c0-6abf57fbbc5d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:57.866872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13699d0-6c4b-4614-81cc-a64e30dcfcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:57.868128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13699d0-6c4b-4614-81cc-a64e30dcfcfb None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:57.868612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13699d0-6c4b-4614-81cc-a64e30dcfcfb None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:57.869336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13699d0-6c4b-4614-81cc-a64e30dcfcfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:57.880317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abd178-b93d-4052-b8fe-1c1356317fbd None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:57.884831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13699d0-6c4b-4614-81cc-a64e30dcfcfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:57.886610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13699d0-6c4b-4614-81cc-a64e30dcfcfb None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:57.893661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-c13699d0-6c4b-4614-81cc-a64e30dcfcfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:32:57.907538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:57.910219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:32:57.911230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:32:57.911230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:32:57.911230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:57.911230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 447/888] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:57.916183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13699d0-6c4b-4614-81cc-a64e30dcfcfb None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:57.925323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707a058-c1b5-4fc2-939c-77ec6884331f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:57.926297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707a058-c1b5-4fc2-939c-77ec6884331f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:57.926800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707a058-c1b5-4fc2-939c-77ec6884331f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:57.927269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707a058-c1b5-4fc2-939c-77ec6884331f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:57.928051 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707a058-c1b5-4fc2-939c-77ec6884331f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:57.929670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707a058-c1b5-4fc2-939c-77ec6884331f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-f707a058-c1b5-4fc2-939c-77ec6884331f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:57.939514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:57.940531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:32:57.941434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:32:57.942822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:57.945093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 442/889] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:32:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:32:57.951046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707a058-c1b5-4fc2-939c-77ec6884331f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:58.022771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13699d0-6c4b-4614-81cc-a64e30dcfcfb None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:58.058488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35690bc-e2e5-4cc8-92a2-188d8e4f0441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:58.059083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35690bc-e2e5-4cc8-92a2-188d8e4f0441 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:58.059626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35690bc-e2e5-4cc8-92a2-188d8e4f0441 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:58.059737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35690bc-e2e5-4cc8-92a2-188d8e4f0441 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:58.060361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35690bc-e2e5-4cc8-92a2-188d8e4f0441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:58.060672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35690bc-e2e5-4cc8-92a2-188d8e4f0441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:58.062156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35690bc-e2e5-4cc8-92a2-188d8e4f0441 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:58.063315 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707a058-c1b5-4fc2-939c-77ec6884331f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:58.074977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 448/890] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:58.080861 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35690bc-e2e5-4cc8-92a2-188d8e4f0441 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:58.110728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aa4e4-d7f2-4a0d-bf48-f8e953176eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:58.111543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aa4e4-d7f2-4a0d-bf48-f8e953176eed None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:58.112009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aa4e4-d7f2-4a0d-bf48-f8e953176eed None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:58.112375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aa4e4-d7f2-4a0d-bf48-f8e953176eed None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:58.113111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0aa4e4-d7f2-4a0d-bf48-f8e953176eed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:58.113509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0aa4e4-d7f2-4a0d-bf48-f8e953176eed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:58.115972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0aa4e4-d7f2-4a0d-bf48-f8e953176eed None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:58.128343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 443/891] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Aug 9 16:32:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:58.129436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aa4e4-d7f2-4a0d-bf48-f8e953176eed None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:58.176850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:58.199324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:58.218832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35690bc-e2e5-4cc8-92a2-188d8e4f0441 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:58.246666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aa4e4-d7f2-4a0d-bf48-f8e953176eed None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:58.266568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6acf43-4004-4aed-bd07-c7e6d7a23692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:58.267267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6acf43-4004-4aed-bd07-c7e6d7a23692 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:58.267613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6acf43-4004-4aed-bd07-c7e6d7a23692 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:58.267950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6acf43-4004-4aed-bd07-c7e6d7a23692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:58.276843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6acf43-4004-4aed-bd07-c7e6d7a23692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:58.278995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6acf43-4004-4aed-bd07-c7e6d7a23692 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:58.285128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512af77d-dc9b-45ce-887a-d594c3dc2cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:58.286055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512af77d-dc9b-45ce-887a-d594c3dc2cff None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:58.286547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512af77d-dc9b-45ce-887a-d594c3dc2cff None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:58.286994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512af77d-dc9b-45ce-887a-d594c3dc2cff None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:58.287861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512af77d-dc9b-45ce-887a-d594c3dc2cff None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:58.288357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512af77d-dc9b-45ce-887a-d594c3dc2cff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:58.290682 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512af77d-dc9b-45ce-887a-d594c3dc2cff None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-7c6acf43-4004-4aed-bd07-c7e6d7a23692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:32:58.293496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:58.294596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:58.295686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:32:58.295686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:32:58.295686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:32:58.295686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:32:58.295686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:32:58.295686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:32:58.295686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:32:58.295686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:58.296425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 449/892] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:32:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:58.303534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6acf43-4004-4aed-bd07-c7e6d7a23692 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:58.303781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 444/893] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Aug 9 16:32:58 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:58.423873 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6acf43-4004-4aed-bd07-c7e6d7a23692 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:58.461497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0cdd8-a741-4823-aa9a-6b6101e4e41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:58.462089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0cdd8-a741-4823-aa9a-6b6101e4e41b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:58.462364 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0cdd8-a741-4823-aa9a-6b6101e4e41b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:58.462626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0cdd8-a741-4823-aa9a-6b6101e4e41b None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:58.463310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0cdd8-a741-4823-aa9a-6b6101e4e41b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:58.463719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0cdd8-a741-4823-aa9a-6b6101e4e41b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:58.465145 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0cdd8-a741-4823-aa9a-6b6101e4e41b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:58.478694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 450/894] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:32:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:58.484681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512af77d-dc9b-45ce-887a-d594c3dc2cff None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:58.595415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512af77d-dc9b-45ce-887a-d594c3dc2cff None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:58.598274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:58.632019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef661a-9e76-4f78-be90-71f95562e981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:58.633081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef661a-9e76-4f78-be90-71f95562e981 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:58.633812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef661a-9e76-4f78-be90-71f95562e981 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:58.634113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef661a-9e76-4f78-be90-71f95562e981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:58.634840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef661a-9e76-4f78-be90-71f95562e981 None admin] RBAC: Adding query filter params (role.id=f4cfecbeb0f44d2aa4a275e60038fa7c, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=a010ea3529644fca951a7eef28dbe97e) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:58.635143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef661a-9e76-4f78-be90-71f95562e981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:58.636650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef661a-9e76-4f78-be90-71f95562e981 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:58.642921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 445/895] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:32:58 2022] GET /identity/v3/role_assignments?user.id=a010ea3529644fca951a7eef28dbe97e&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=f4cfecbeb0f44d2aa4a275e60038fa7c => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:59.019372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545fc57-96ad-4641-97c5-ce650410f852 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:59.019755 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545fc57-96ad-4641-97c5-ce650410f852 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:59.019938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545fc57-96ad-4641-97c5-ce650410f852 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:59.021513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 451/896] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:32:59.027546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa15de-2dc0-49c5-be44-6975793e528d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:59.027838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa15de-2dc0-49c5-be44-6975793e528d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:59.028107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa15de-2dc0-49c5-be44-6975793e528d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:59.047078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0bfa15de-2dc0-49c5-be44-6975793e528d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:59.048270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0bfa15de-2dc0-49c5-be44-6975793e528d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:59.121613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 446/897] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:32:59.144906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545fc57-96ad-4641-97c5-ce650410f852 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:59.149496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:59.249547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545fc57-96ad-4641-97c5-ce650410f852 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:59.288453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1484a7d-8c2b-495e-a3a7-0bb26d41dcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:59.289028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1484a7d-8c2b-495e-a3a7-0bb26d41dcb5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:59.289300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1484a7d-8c2b-495e-a3a7-0bb26d41dcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:59.289568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1484a7d-8c2b-495e-a3a7-0bb26d41dcb5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:59.290178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1484a7d-8c2b-495e-a3a7-0bb26d41dcb5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:59.291529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1484a7d-8c2b-495e-a3a7-0bb26d41dcb5 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-b1484a7d-8c2b-495e-a3a7-0bb26d41dcb5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:32:59.300464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:32:59.301468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:32:59.302333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 452/898] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:32:59.308476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa15de-2dc0-49c5-be44-6975793e528d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:59.405340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:59.411451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa15de-2dc0-49c5-be44-6975793e528d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:59.448497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e17e9d-b71a-4c82-a483-ed0bf4707ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:59.449111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e17e9d-b71a-4c82-a483-ed0bf4707ce4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:59.449386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e17e9d-b71a-4c82-a483-ed0bf4707ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:59.449652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e17e9d-b71a-4c82-a483-ed0bf4707ce4 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:59.450332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e17e9d-b71a-4c82-a483-ed0bf4707ce4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:59.450634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e17e9d-b71a-4c82-a483-ed0bf4707ce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:59.451998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e17e9d-b71a-4c82-a483-ed0bf4707ce4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:59.462189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 447/899] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:59.463802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1484a7d-8c2b-495e-a3a7-0bb26d41dcb5 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:59.535812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:59.571344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1484a7d-8c2b-495e-a3a7-0bb26d41dcb5 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:59.610419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b326e9-a7ab-440f-89f6-ca2f7dfe5b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:59.611319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b326e9-a7ab-440f-89f6-ca2f7dfe5b80 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:59.611806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b326e9-a7ab-440f-89f6-ca2f7dfe5b80 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:59.612276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b326e9-a7ab-440f-89f6-ca2f7dfe5b80 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:59.613075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b326e9-a7ab-440f-89f6-ca2f7dfe5b80 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:59.613547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b326e9-a7ab-440f-89f6-ca2f7dfe5b80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:59.615233 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b326e9-a7ab-440f-89f6-ca2f7dfe5b80 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:59.625655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 453/900] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:32:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:32:59.627809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e17e9d-b71a-4c82-a483-ed0bf4707ce4 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:59.730664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e17e9d-b71a-4c82-a483-ed0bf4707ce4 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:59.765237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40efe0-9234-47ad-9be9-a26b97cb810d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:59.766165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40efe0-9234-47ad-9be9-a26b97cb810d None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:59.766621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40efe0-9234-47ad-9be9-a26b97cb810d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:59.767092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40efe0-9234-47ad-9be9-a26b97cb810d None admin] PATH_INFO: `/v3/endpoints` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:59.767957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40efe0-9234-47ad-9be9-a26b97cb810d None admin] RBAC: Adding query filter params (interface=public, service_id=1c08ea26fd424d17b1e45d0b8bebea24, region_id=RegionOne) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:32:59.768439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40efe0-9234-47ad-9be9-a26b97cb810d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:59.769987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40efe0-9234-47ad-9be9-a26b97cb810d None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:59.780353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 448/901] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Aug 9 16:32:59 2022] GET /identity/v3/endpoints?service_id=1c08ea26fd424d17b1e45d0b8bebea24&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:32:59.781939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b326e9-a7ab-440f-89f6-ca2f7dfe5b80 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:59.785584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e4513-daef-4f90-b3b8-5649e27253a4 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:59.785872 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e4513-daef-4f90-b3b8-5649e27253a4 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:59.786177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e4513-daef-4f90-b3b8-5649e27253a4 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:59.787610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 449/902] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:32:59.792808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5abb4e-d1ae-4fcb-8a2c-976dcb0d5dcd None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:59.793089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5abb4e-d1ae-4fcb-8a2c-976dcb0d5dcd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:59.793365 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5abb4e-d1ae-4fcb-8a2c-976dcb0d5dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:59.814003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0c5abb4e-d1ae-4fcb-8a2c-976dcb0d5dcd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:32:59.814003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0c5abb4e-d1ae-4fcb-8a2c-976dcb0d5dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:32:59.835437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:59.902746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b326e9-a7ab-440f-89f6-ca2f7dfe5b80 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:32:59.906111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 450/903] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:32:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:32:59.927142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5abb4e-d1ae-4fcb-8a2c-976dcb0d5dcd None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:32:59.945704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:32:59.952808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3ef87-6990-41d8-9542-c994f93135d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:32:59.954011 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3ef87-6990-41d8-9542-c994f93135d7 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:32:59.954701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3ef87-6990-41d8-9542-c994f93135d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:32:59.955122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3ef87-6990-41d8-9542-c994f93135d7 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:32:59.956033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3ef87-6990-41d8-9542-c994f93135d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:32:59.958186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf3ef87-6990-41d8-9542-c994f93135d7 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:32:59.966950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 454/904] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Aug 9 16:32:59 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:33:00.039427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5abb4e-d1ae-4fcb-8a2c-976dcb0d5dcd None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:00.078075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec669012-9a3b-4160-a103-aa0dd1485b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:00.078975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec669012-9a3b-4160-a103-aa0dd1485b9a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:00.079398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec669012-9a3b-4160-a103-aa0dd1485b9a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:00.079823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec669012-9a3b-4160-a103-aa0dd1485b9a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:00.080564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec669012-9a3b-4160-a103-aa0dd1485b9a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:00.082190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec669012-9a3b-4160-a103-aa0dd1485b9a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-ec669012-9a3b-4160-a103-aa0dd1485b9a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:00.093030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:00.094233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 16:33:00.095292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:00.097626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 451/905] 10.222.0.79 () {56 vars in 1132 bytes} [Tue Aug 9 16:32:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:00.104228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3ef87-6990-41d8-9542-c994f93135d7 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:00.215654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf3ef87-6990-41d8-9542-c994f93135d7 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:00.260618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26e73e-b757-4927-baef-1575ba85829b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:00.261212 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26e73e-b757-4927-baef-1575ba85829b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:00.261487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26e73e-b757-4927-baef-1575ba85829b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:00.261753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26e73e-b757-4927-baef-1575ba85829b None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:00.262411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26e73e-b757-4927-baef-1575ba85829b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:00.262707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26e73e-b757-4927-baef-1575ba85829b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:00.264069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26e73e-b757-4927-baef-1575ba85829b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:00.269312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 455/906] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:33:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:00.275442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec669012-9a3b-4160-a103-aa0dd1485b9a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:00.298313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:00.392948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec669012-9a3b-4160-a103-aa0dd1485b9a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:00.428733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b2208-109e-4551-bdfe-f90d38ec7b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:00.429743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b2208-109e-4551-bdfe-f90d38ec7b39 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:00.430250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b2208-109e-4551-bdfe-f90d38ec7b39 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:00.430735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b2208-109e-4551-bdfe-f90d38ec7b39 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:00.431555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b2208-109e-4551-bdfe-f90d38ec7b39 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:00.432090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b2208-109e-4551-bdfe-f90d38ec7b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:00.433645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b2208-109e-4551-bdfe-f90d38ec7b39 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:00.439653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 452/907] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:33:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:00.447033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26e73e-b757-4927-baef-1575ba85829b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:00.465589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71c34f-eeb1-419c-8430-fb0c17ba4995 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:00.465924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71c34f-eeb1-419c-8430-fb0c17ba4995 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:00.466168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d71c34f-eeb1-419c-8430-fb0c17ba4995 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:00.467866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 453/908] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:00.472669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc635-f1a6-4573-99b8-14a26a3c6fb4 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:00.472947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc635-f1a6-4573-99b8-14a26a3c6fb4 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:00.473216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc635-f1a6-4573-99b8-14a26a3c6fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:00.491963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0ecfc635-f1a6-4573-99b8-14a26a3c6fb4 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:00.493153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0ecfc635-f1a6-4573-99b8-14a26a3c6fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:00.514152 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:00.560140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26e73e-b757-4927-baef-1575ba85829b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:00.574313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 454/909] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:00.589372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfc635-f1a6-4573-99b8-14a26a3c6fb4 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:00.598520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e2ac4-d37c-47de-b6d5-8894f1e7c8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:00.599457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e2ac4-d37c-47de-b6d5-8894f1e7c8af None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:00.599959 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e2ac4-d37c-47de-b6d5-8894f1e7c8af None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:00.600453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e2ac4-d37c-47de-b6d5-8894f1e7c8af None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:00.601391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e2ac4-d37c-47de-b6d5-8894f1e7c8af None admin] RBAC: Adding query filter params (interface=public, service_id=22755892f53445b283d99a2af99c32fd, region_id=RegionOne) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:00.601906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e2ac4-d37c-47de-b6d5-8894f1e7c8af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:00.603480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e2ac4-d37c-47de-b6d5-8894f1e7c8af None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:00.604694 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:00.613785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 456/910] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Aug 9 16:33:00 2022] GET /identity/v3/endpoints?service_id=22755892f53445b283d99a2af99c32fd&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:00.619498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e2ac4-d37c-47de-b6d5-8894f1e7c8af None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:00.702027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfc635-f1a6-4573-99b8-14a26a3c6fb4 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:00.735753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e2ac4-d37c-47de-b6d5-8894f1e7c8af None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:00.741026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e804f-078f-4ed0-9db5-5fddf13d4fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:00.742159 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e804f-078f-4ed0-9db5-5fddf13d4fd3 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:00.742444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e804f-078f-4ed0-9db5-5fddf13d4fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:00.742712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e804f-078f-4ed0-9db5-5fddf13d4fd3 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:00.743396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e804f-078f-4ed0-9db5-5fddf13d4fd3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:00.744814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e804f-078f-4ed0-9db5-5fddf13d4fd3 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:00.764412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:00.775413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68595071-97d0-42a1-a751-3eba0732ed88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:00.776015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68595071-97d0-42a1-a751-3eba0732ed88 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:00.776290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68595071-97d0-42a1-a751-3eba0732ed88 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:00.776555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68595071-97d0-42a1-a751-3eba0732ed88 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:00.777156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68595071-97d0-42a1-a751-3eba0732ed88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:00.778581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68595071-97d0-42a1-a751-3eba0732ed88 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:00.784448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 457/911] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Aug 9 16:33:00 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:33:00.836134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 455/912] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:33:00 2022] POST /identity/v3/services => generated 241 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:33:00.868056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f5729-a6d2-40b8-98bc-85abc5580984 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:00.868329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f5729-a6d2-40b8-98bc-85abc5580984 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:00.868598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f5729-a6d2-40b8-98bc-85abc5580984 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:00.870356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 458/913] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:00.876320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2769b-5047-42ce-bcbc-ccd578617b2b None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:00.876675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2769b-5047-42ce-bcbc-ccd578617b2b None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:00.876867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2769b-5047-42ce-bcbc-ccd578617b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:00.897444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-83e2769b-5047-42ce-bcbc-ccd578617b2b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:00.898791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-83e2769b-5047-42ce-bcbc-ccd578617b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:00.980976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:00.987780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 456/914] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:01.014744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f5729-a6d2-40b8-98bc-85abc5580984 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:01.131547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f5729-a6d2-40b8-98bc-85abc5580984 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:01.166912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8654b2f-deda-4b2c-9512-f4b1a656758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:01.167534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8654b2f-deda-4b2c-9512-f4b1a656758b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:01.167806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8654b2f-deda-4b2c-9512-f4b1a656758b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:01.168074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8654b2f-deda-4b2c-9512-f4b1a656758b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:01.179981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8654b2f-deda-4b2c-9512-f4b1a656758b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-a8654b2f-deda-4b2c-9512-f4b1a656758b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:01.184778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:01.185684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:33:01.186550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:01.190346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 459/915] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Aug 9 16:33:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:01.192248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2769b-5047-42ce-bcbc-ccd578617b2b None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:01.299129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2769b-5047-42ce-bcbc-ccd578617b2b None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:01.334151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea048e0-74ca-4d57-98bb-a974d5a00c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:01.334753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea048e0-74ca-4d57-98bb-a974d5a00c67 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:01.335031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea048e0-74ca-4d57-98bb-a974d5a00c67 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:01.335296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea048e0-74ca-4d57-98bb-a974d5a00c67 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:01.335981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea048e0-74ca-4d57-98bb-a974d5a00c67 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:01.336276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea048e0-74ca-4d57-98bb-a974d5a00c67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:01.337628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea048e0-74ca-4d57-98bb-a974d5a00c67 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:01.347913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 457/916] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Aug 9 16:33:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:01.354100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8654b2f-deda-4b2c-9512-f4b1a656758b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:01.407836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:01.459971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8654b2f-deda-4b2c-9512-f4b1a656758b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:01.495275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaa566-dbae-44af-83bc-53ae52a66b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:01.495897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa566-dbae-44af-83bc-53ae52a66b56 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:01.496173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa566-dbae-44af-83bc-53ae52a66b56 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:01.496456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaa566-dbae-44af-83bc-53ae52a66b56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:01.513200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaa566-dbae-44af-83bc-53ae52a66b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:01.514839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaa566-dbae-44af-83bc-53ae52a66b56 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-d7aaa566-dbae-44af-83bc-53ae52a66b56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:01.535100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:01.537282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:01.540097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:01.540097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:01.540097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:01.540097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 460/917] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:01.543311 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea048e0-74ca-4d57-98bb-a974d5a00c67 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:01.611140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:01.656809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea048e0-74ca-4d57-98bb-a974d5a00c67 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:01.691670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb7c49-a140-463e-838c-cbf3800b268e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:01.692317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb7c49-a140-463e-838c-cbf3800b268e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:01.692597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb7c49-a140-463e-838c-cbf3800b268e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:01.692866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb7c49-a140-463e-838c-cbf3800b268e None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:01.693554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb7c49-a140-463e-838c-cbf3800b268e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:01.693857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb7c49-a140-463e-838c-cbf3800b268e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:01.695271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb7c49-a140-463e-838c-cbf3800b268e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:01.707536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 458/918] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:01.714029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaa566-dbae-44af-83bc-53ae52a66b56 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:01.816417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:01.819527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaa566-dbae-44af-83bc-53ae52a66b56 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:01.853505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c272c5-7835-415b-8ab7-9d0648ca6369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:01.854148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c272c5-7835-415b-8ab7-9d0648ca6369 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:01.854773 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c272c5-7835-415b-8ab7-9d0648ca6369 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:01.855568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c272c5-7835-415b-8ab7-9d0648ca6369 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:01.863163 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c272c5-7835-415b-8ab7-9d0648ca6369 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:01.864523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c272c5-7835-415b-8ab7-9d0648ca6369 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-a9c272c5-7835-415b-8ab7-9d0648ca6369 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:33:01.875554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:01.876511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:01.877460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 461/919] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Aug 9 16:33:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:01.882956 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb7c49-a140-463e-838c-cbf3800b268e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:01.988630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb7c49-a140-463e-838c-cbf3800b268e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:02.009381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:02.024086 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81d2bb-c096-465b-a2f6-e4b86ac16f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:02.024680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81d2bb-c096-465b-a2f6-e4b86ac16f80 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.024950 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81d2bb-c096-465b-a2f6-e4b86ac16f80 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.025212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81d2bb-c096-465b-a2f6-e4b86ac16f80 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.025997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81d2bb-c096-465b-a2f6-e4b86ac16f80 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:02.026299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81d2bb-c096-465b-a2f6-e4b86ac16f80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:02.027820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81d2bb-c096-465b-a2f6-e4b86ac16f80 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:02.047069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 459/920] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Aug 9 16:33:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:02.055612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c272c5-7835-415b-8ab7-9d0648ca6369 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:02.122798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22f365-ef58-4989-99e3-ea8dbb4a38f4 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.123005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22f365-ef58-4989-99e3-ea8dbb4a38f4 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.123241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22f365-ef58-4989-99e3-ea8dbb4a38f4 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.125002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 460/921] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:02.130662 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7594e-755f-4b84-8b6b-c8379d53be73 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.130940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7594e-755f-4b84-8b6b-c8379d53be73 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.131213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec7594e-755f-4b84-8b6b-c8379d53be73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.150910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6ec7594e-755f-4b84-8b6b-c8379d53be73 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:02.152094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6ec7594e-755f-4b84-8b6b-c8379d53be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:02.166733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c272c5-7835-415b-8ab7-9d0648ca6369 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:02.208473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d28d04-3949-49b4-9ed5-2cb328553ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:02.209615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d28d04-3949-49b4-9ed5-2cb328553ab0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.210511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d28d04-3949-49b4-9ed5-2cb328553ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.211058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d28d04-3949-49b4-9ed5-2cb328553ab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.221545 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 461/922] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:02.225504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d28d04-3949-49b4-9ed5-2cb328553ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:02.227160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d28d04-3949-49b4-9ed5-2cb328553ab0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-82d28d04-3949-49b4-9ed5-2cb328553ab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:02.242279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:02.243439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:02.244517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:02.244517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:02.244517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:02.245650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7594e-755f-4b84-8b6b-c8379d53be73 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:02.246408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 462/923] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:02.251900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d28d04-3949-49b4-9ed5-2cb328553ab0 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:02.269267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:02.352606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec7594e-755f-4b84-8b6b-c8379d53be73 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:02.362054 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:02.367954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d28d04-3949-49b4-9ed5-2cb328553ab0 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:02.388049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1050ebf-5716-4271-a99e-88b46ef493c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:02.388677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1050ebf-5716-4271-a99e-88b46ef493c7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.389027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1050ebf-5716-4271-a99e-88b46ef493c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.389356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1050ebf-5716-4271-a99e-88b46ef493c7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.390102 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1050ebf-5716-4271-a99e-88b46ef493c7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:02.391563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1050ebf-5716-4271-a99e-88b46ef493c7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-d1050ebf-5716-4271-a99e-88b46ef493c7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:02.400917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:02.402177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 09 16:33:02.403101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:02.407474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 462/924] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:33:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:02.408770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1050ebf-5716-4271-a99e-88b46ef493c7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:02.409421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c69d5f-878a-4891-8477-bd54f7cc3a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:02.410214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69d5f-878a-4891-8477-bd54f7cc3a2d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.410657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69d5f-878a-4891-8477-bd54f7cc3a2d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.411072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69d5f-878a-4891-8477-bd54f7cc3a2d None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.411907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c69d5f-878a-4891-8477-bd54f7cc3a2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:02.412434 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c69d5f-878a-4891-8477-bd54f7cc3a2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:02.414073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c69d5f-878a-4891-8477-bd54f7cc3a2d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:02.427428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 463/925] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:02.438009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c69d5f-878a-4891-8477-bd54f7cc3a2d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:02.535038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1050ebf-5716-4271-a99e-88b46ef493c7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:02.540089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:02.572267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39befb-1657-4cea-9327-5cd6fc5b8b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:02.572863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39befb-1657-4cea-9327-5cd6fc5b8b48 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.573136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39befb-1657-4cea-9327-5cd6fc5b8b48 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.573399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39befb-1657-4cea-9327-5cd6fc5b8b48 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.574157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39befb-1657-4cea-9327-5cd6fc5b8b48 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:02.574435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39befb-1657-4cea-9327-5cd6fc5b8b48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:02.575835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39befb-1657-4cea-9327-5cd6fc5b8b48 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:02.583046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c69d5f-878a-4891-8477-bd54f7cc3a2d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:02.585621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 463/926] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:33:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:02.587002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39befb-1657-4cea-9327-5cd6fc5b8b48 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:02.625757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:02.632311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c5e47-4823-4fb6-a6ed-57cb1a648a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:02.633545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c5e47-4823-4fb6-a6ed-57cb1a648a40 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.634353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c5e47-4823-4fb6-a6ed-57cb1a648a40 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.635050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c5e47-4823-4fb6-a6ed-57cb1a648a40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.647249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c5e47-4823-4fb6-a6ed-57cb1a648a40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:02.647585 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c5e47-4823-4fb6-a6ed-57cb1a648a40 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-fb3c5e47-4823-4fb6-a6ed-57cb1a648a40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:33:02.661349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:02.662303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:02.663209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:33:02.663209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:33:02.663209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:33:02.663209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:33:02.663209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:33:02.663209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:33:02.663209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:02.663209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:02.668214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 464/927] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:33:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:02.669160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c5e47-4823-4fb6-a6ed-57cb1a648a40 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:02.709328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39befb-1657-4cea-9327-5cd6fc5b8b48 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:02.745660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d58909-3989-4526-aed6-bdd84ef066eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:02.746299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d58909-3989-4526-aed6-bdd84ef066eb None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.746571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d58909-3989-4526-aed6-bdd84ef066eb None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.746886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d58909-3989-4526-aed6-bdd84ef066eb None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.747505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d58909-3989-4526-aed6-bdd84ef066eb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:02.747798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d58909-3989-4526-aed6-bdd84ef066eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:02.749133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d58909-3989-4526-aed6-bdd84ef066eb None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:02.759721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 464/928] 10.222.0.79 () {56 vars in 1114 bytes} [Tue Aug 9 16:33:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:02.761534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d58909-3989-4526-aed6-bdd84ef066eb None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:02.795877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c5e47-4823-4fb6-a6ed-57cb1a648a40 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:02.834113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:02.841031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71961518-1803-4577-8644-a04adcab4a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:02.842245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71961518-1803-4577-8644-a04adcab4a99 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.842664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71961518-1803-4577-8644-a04adcab4a99 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.843073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71961518-1803-4577-8644-a04adcab4a99 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.844336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71961518-1803-4577-8644-a04adcab4a99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:02.844807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71961518-1803-4577-8644-a04adcab4a99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:02.846757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71961518-1803-4577-8644-a04adcab4a99 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:02.864333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 465/929] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:33:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:02.870404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:02.871288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71961518-1803-4577-8644-a04adcab4a99 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:02.873893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d58909-3989-4526-aed6-bdd84ef066eb None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:02.909977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77401826-4cdc-47a1-aeab-fca5af030ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:02.910570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77401826-4cdc-47a1-aeab-fca5af030ce8 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:02.910769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77401826-4cdc-47a1-aeab-fca5af030ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:02.911072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77401826-4cdc-47a1-aeab-fca5af030ce8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:02.911725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77401826-4cdc-47a1-aeab-fca5af030ce8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:02.913194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77401826-4cdc-47a1-aeab-fca5af030ce8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:03.002986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71961518-1803-4577-8644-a04adcab4a99 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:03.003610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 465/930] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:33:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:33:03.003610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cf84a-071f-4545-9b84-2f968a8bb8d3 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.003610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cf84a-071f-4545-9b84-2f968a8bb8d3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.003610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cf84a-071f-4545-9b84-2f968a8bb8d3 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.003610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 466/931] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:03.003610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962a819-aa62-4d32-b872-c9baffb4c420 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.004000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962a819-aa62-4d32-b872-c9baffb4c420 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.004271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962a819-aa62-4d32-b872-c9baffb4c420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.042156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2962a819-aa62-4d32-b872-c9baffb4c420 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:03.042156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2962a819-aa62-4d32-b872-c9baffb4c420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:03.078022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7329fc-c475-411d-a151-e27e318556e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:03.078022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7329fc-c475-411d-a151-e27e318556e8 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.081987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7329fc-c475-411d-a151-e27e318556e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.081987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7329fc-c475-411d-a151-e27e318556e8 None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/a010ea3529644fca951a7eef28dbe97e/roles/f4cfecbeb0f44d2aa4a275e60038fa7c` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.082276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7329fc-c475-411d-a151-e27e318556e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=a010ea3529644fca951a7eef28dbe97e, role_id=f4cfecbeb0f44d2aa4a275e60038fa7c)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:03.084791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7329fc-c475-411d-a151-e27e318556e8 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:03.127516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 467/932] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:03.129087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcbca0-258b-4aca-be63-6717fa7c4e31 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.129377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcbca0-258b-4aca-be63-6717fa7c4e31 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.129650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcbca0-258b-4aca-be63-6717fa7c4e31 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.131064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 468/933] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:03.136760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc68a4a-7890-4687-9a11-c0c176eded73 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.137044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc68a4a-7890-4687-9a11-c0c176eded73 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.137316 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc68a4a-7890-4687-9a11-c0c176eded73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.156713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-cbc68a4a-7890-4687-9a11-c0c176eded73 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:03.157942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-cbc68a4a-7890-4687-9a11-c0c176eded73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:03.207748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 466/934] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:33:02 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/a010ea3529644fca951a7eef28dbe97e/roles/f4cfecbeb0f44d2aa4a275e60038fa7c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:33:03.208677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7329fc-c475-411d-a151-e27e318556e8 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:03.252861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 469/935] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:03.275519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc68a4a-7890-4687-9a11-c0c176eded73 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:03.280125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:03.304004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:03.348364 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7329fc-c475-411d-a151-e27e318556e8 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:03.389465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332490d9-934c-4f7b-8422-046ffe0404e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:03.390442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332490d9-934c-4f7b-8422-046ffe0404e5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.390442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332490d9-934c-4f7b-8422-046ffe0404e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.394787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332490d9-934c-4f7b-8422-046ffe0404e5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.394787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332490d9-934c-4f7b-8422-046ffe0404e5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:03.394787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332490d9-934c-4f7b-8422-046ffe0404e5 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-332490d9-934c-4f7b-8422-046ffe0404e5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:03.408182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:03.409235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:03.410255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 467/936] 10.222.0.79 () {56 vars in 1132 bytes} [Tue Aug 9 16:33:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:03.413215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc68a4a-7890-4687-9a11-c0c176eded73 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:03.419168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332490d9-934c-4f7b-8422-046ffe0404e5 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:03.459484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029dc8a-3cc9-45d9-9942-748d19d2c652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:03.460462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029dc8a-3cc9-45d9-9942-748d19d2c652 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.461424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029dc8a-3cc9-45d9-9942-748d19d2c652 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.461866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029dc8a-3cc9-45d9-9942-748d19d2c652 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.462652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029dc8a-3cc9-45d9-9942-748d19d2c652 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:03.464963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029dc8a-3cc9-45d9-9942-748d19d2c652 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-8029dc8a-3cc9-45d9-9942-748d19d2c652 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:03.478527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:03.479832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:03.480959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 470/937] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:33:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:03.488140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029dc8a-3cc9-45d9-9942-748d19d2c652 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:03.554410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332490d9-934c-4f7b-8422-046ffe0404e5 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:03.571052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:03.600759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd5345-df2e-49bd-9ac5-60189ed9a970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:03.601546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd5345-df2e-49bd-9ac5-60189ed9a970 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.601911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd5345-df2e-49bd-9ac5-60189ed9a970 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.602305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd5345-df2e-49bd-9ac5-60189ed9a970 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.603055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd5345-df2e-49bd-9ac5-60189ed9a970 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:03.603456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd5345-df2e-49bd-9ac5-60189ed9a970 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:03.605097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd5345-df2e-49bd-9ac5-60189ed9a970 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:03.610508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:03.611520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 468/938] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:33:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:03.617254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd5345-df2e-49bd-9ac5-60189ed9a970 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:03.644853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029dc8a-3cc9-45d9-9942-748d19d2c652 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:03.689206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e57eb-57d2-4020-b171-a1b578028941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:03.690016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e57eb-57d2-4020-b171-a1b578028941 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.690460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e57eb-57d2-4020-b171-a1b578028941 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.690836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e57eb-57d2-4020-b171-a1b578028941 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.691556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e57eb-57d2-4020-b171-a1b578028941 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:03.691958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e57eb-57d2-4020-b171-a1b578028941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:03.693489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98e57eb-57d2-4020-b171-a1b578028941 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:03.700191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 471/939] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Aug 9 16:33:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:03.705891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e57eb-57d2-4020-b171-a1b578028941 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:03.780867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd5345-df2e-49bd-9ac5-60189ed9a970 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:03.819231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85c872-0a6b-4bf3-96a9-d01030cafd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:03.820913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85c872-0a6b-4bf3-96a9-d01030cafd14 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.820913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85c872-0a6b-4bf3-96a9-d01030cafd14 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.821245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85c872-0a6b-4bf3-96a9-d01030cafd14 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.822012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85c872-0a6b-4bf3-96a9-d01030cafd14 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:03.822436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85c872-0a6b-4bf3-96a9-d01030cafd14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:03.824181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85c872-0a6b-4bf3-96a9-d01030cafd14 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:03.835957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 469/940] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:33:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:03.837856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85c872-0a6b-4bf3-96a9-d01030cafd14 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:03.863892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e57eb-57d2-4020-b171-a1b578028941 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:03.908917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:03.915221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2d494-babf-4854-9bd7-30ac2e9d5e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:03.915837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2d494-babf-4854-9bd7-30ac2e9d5e6f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:03.916112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2d494-babf-4854-9bd7-30ac2e9d5e6f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:03.916381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2d494-babf-4854-9bd7-30ac2e9d5e6f None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:03.917427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2d494-babf-4854-9bd7-30ac2e9d5e6f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:03.917738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2d494-babf-4854-9bd7-30ac2e9d5e6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:03.919543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d2d494-babf-4854-9bd7-30ac2e9d5e6f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:03.927873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 472/941] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Aug 9 16:33:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:03.932674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2d494-babf-4854-9bd7-30ac2e9d5e6f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:03.961727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:04.002864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85c872-0a6b-4bf3-96a9-d01030cafd14 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:04.042239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2d494-babf-4854-9bd7-30ac2e9d5e6f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:04.048826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54211316-6147-4975-a025-b8482da07745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:04.049713 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54211316-6147-4975-a025-b8482da07745 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:04.050601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54211316-6147-4975-a025-b8482da07745 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:04.051039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54211316-6147-4975-a025-b8482da07745 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:04.051878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54211316-6147-4975-a025-b8482da07745 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:04.054867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54211316-6147-4975-a025-b8482da07745 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:04.086625 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd48743-bd5a-4afb-99bc-1c4ab7995aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:04.087231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd48743-bd5a-4afb-99bc-1c4ab7995aa9 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:04.087549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd48743-bd5a-4afb-99bc-1c4ab7995aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:04.087825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd48743-bd5a-4afb-99bc-1c4ab7995aa9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:04.088583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd48743-bd5a-4afb-99bc-1c4ab7995aa9 None admin] RBAC: Adding query filter params (interface=public, service_id=f4f7e1bf31ff404287c39408a211dd64, region_id=RegionOne) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:04.088941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd48743-bd5a-4afb-99bc-1c4ab7995aa9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:04.090327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd48743-bd5a-4afb-99bc-1c4ab7995aa9 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:04.096691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 473/942] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Aug 9 16:33:03 2022] GET /identity/v3/endpoints?service_id=f4f7e1bf31ff404287c39408a211dd64&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:04.103046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd48743-bd5a-4afb-99bc-1c4ab7995aa9 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:04.166434 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 470/943] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:33:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:33:04.201234 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:04.243922 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd48743-bd5a-4afb-99bc-1c4ab7995aa9 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:04.296447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af84db6c-7ee4-4b43-8162-a558010db73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:04.297318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84db6c-7ee4-4b43-8162-a558010db73e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:04.297746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84db6c-7ee4-4b43-8162-a558010db73e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:04.298263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af84db6c-7ee4-4b43-8162-a558010db73e None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:04.299030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84db6c-7ee4-4b43-8162-a558010db73e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:04.300571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af84db6c-7ee4-4b43-8162-a558010db73e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:04.308138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 474/944] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Aug 9 16:33:04 2022] GET /identity/v3/services => generated 1497 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:33:05.270972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b8fab-0386-4dc4-86ed-a855259f9145 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:05.271531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b8fab-0386-4dc4-86ed-a855259f9145 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:05.271531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b8fab-0386-4dc4-86ed-a855259f9145 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:05.273641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 471/945] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:05.279313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81372-fcf2-4915-9afc-2d4373ddbf22 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:05.280143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81372-fcf2-4915-9afc-2d4373ddbf22 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:05.280592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81372-fcf2-4915-9afc-2d4373ddbf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:05.299753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d7b81372-fcf2-4915-9afc-2d4373ddbf22 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:05.301388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d7b81372-fcf2-4915-9afc-2d4373ddbf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:05.376644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 475/946] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:05.400436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b8fab-0386-4dc4-86ed-a855259f9145 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:05.418182 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7962c3-da1a-4123-9eb9-9b6a21fa04d1 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:05.419105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7962c3-da1a-4123-9eb9-9b6a21fa04d1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:05.419595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7962c3-da1a-4123-9eb9-9b6a21fa04d1 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:05.422041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 476/947] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:05.428331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06302e0b-416a-465f-abad-dbc46cd5d39b None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:05.429060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06302e0b-416a-465f-abad-dbc46cd5d39b None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:05.430433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06302e0b-416a-465f-abad-dbc46cd5d39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:05.456834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-06302e0b-416a-465f-abad-dbc46cd5d39b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:05.459186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-06302e0b-416a-465f-abad-dbc46cd5d39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:05.519493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:05.530060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b8fab-0386-4dc4-86ed-a855259f9145 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:05.543804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:05.559798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 477/948] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:05.574527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:05.576923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:05.576923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:05.576923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:05.576923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:05.578572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:05.581463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: WARNING keystone.server.flask.common [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 09 16:33:05.588847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06302e0b-416a-465f-abad-dbc46cd5d39b None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:05.633127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.sql.core [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 16:33:05.633127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 16:33:05.633127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [parameters: {'id': '02fa77ea07664ea3930c640d219c45b3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 16:33:05.633127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86843) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 02fa77ea07664ea3930c640d219c45b3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 02fa77ea07664ea3930c640d219c45b3. Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application conn.query(q) Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 16:33:05.662534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application result.read() Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 09 16:33:05.663439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application next(self.gen) Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application yield resource Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application next(self.gen) Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise self.value Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application session.commit() Aug 09 16:33:05.664960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self.session.flush() Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application compat.raise_( Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 16:33:05.665834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 16:33:05.667399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application conn.query(q) Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application result.read() Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 16:33:05.675388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [parameters: {'id': '02fa77ea07664ea3930c640d219c45b3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:05.678444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:05.681746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 02fa77ea07664ea3930c640d219c45b3. Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 472/949] 10.222.0.79 () {60 vars in 1140 bytes} [Tue Aug 9 16:33:05 2022] POST /identity/v3/projects => generated 262 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:05.683983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:05.713680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06302e0b-416a-465f-abad-dbc46cd5d39b None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:05.763214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ddd9c-af0e-45b2-a2ab-5d599ce2c450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:05.764455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ddd9c-af0e-45b2-a2ab-5d599ce2c450 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:05.765219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ddd9c-af0e-45b2-a2ab-5d599ce2c450 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:05.766099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ddd9c-af0e-45b2-a2ab-5d599ce2c450 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:05.781686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ddd9c-af0e-45b2-a2ab-5d599ce2c450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-0c1ddd9c-af0e-45b2-a2ab-5d599ce2c450 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:05.786768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:05.787766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:33:05.789134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:05.793639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 478/950] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Aug 9 16:33:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:05.794604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ddd9c-af0e-45b2-a2ab-5d599ce2c450 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:05.820479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fe4c4-f748-4698-ba73-03351dc0b00d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:05.856615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:05.866563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32999d-ce11-47f9-bb7e-6c717112aa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:05.867968 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32999d-ce11-47f9-bb7e-6c717112aa5b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:05.868558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32999d-ce11-47f9-bb7e-6c717112aa5b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:05.869322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32999d-ce11-47f9-bb7e-6c717112aa5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:05.880571 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32999d-ce11-47f9-bb7e-6c717112aa5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:05.882818 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32999d-ce11-47f9-bb7e-6c717112aa5b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-0f32999d-ce11-47f9-bb7e-6c717112aa5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:33:05.895021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:05.896167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:05.897281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:33:05.897281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:33:05.897281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:33:05.897281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:33:05.897281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:33:05.897281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:33:05.897281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:05.897281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:05.898474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 473/951] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:33:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:05.906025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32999d-ce11-47f9-bb7e-6c717112aa5b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:05.920333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:05.930990 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ddd9c-af0e-45b2-a2ab-5d599ce2c450 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:05.991281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7d7c-69e9-4ee3-aafa-1a99c6cca484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:05.992603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7d7c-69e9-4ee3-aafa-1a99c6cca484 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:05.992603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7d7c-69e9-4ee3-aafa-1a99c6cca484 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:05.992980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f7d7c-69e9-4ee3-aafa-1a99c6cca484 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:05.993807 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f7d7c-69e9-4ee3-aafa-1a99c6cca484 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:05.994104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f7d7c-69e9-4ee3-aafa-1a99c6cca484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:05.995597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f7d7c-69e9-4ee3-aafa-1a99c6cca484 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:06.009345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 479/952] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Aug 9 16:33:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:06.010860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7d7c-69e9-4ee3-aafa-1a99c6cca484 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:06.012436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32999d-ce11-47f9-bb7e-6c717112aa5b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:06.056840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4d7e7-eb81-4d23-b331-937048b9e415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:06.057786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4d7e7-eb81-4d23-b331-937048b9e415 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:06.058264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4d7e7-eb81-4d23-b331-937048b9e415 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:06.058743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e4d7e7-eb81-4d23-b331-937048b9e415 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:06.059570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4d7e7-eb81-4d23-b331-937048b9e415 None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:06.060056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4d7e7-eb81-4d23-b331-937048b9e415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:06.061749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e4d7e7-eb81-4d23-b331-937048b9e415 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:06.091004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 474/953] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:33:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:06.139402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f7d7c-69e9-4ee3-aafa-1a99c6cca484 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:06.157319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:06.183747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d5a63-42c3-47ef-bffc-c21769a06b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:06.184721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d5a63-42c3-47ef-bffc-c21769a06b92 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:06.185179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d5a63-42c3-47ef-bffc-c21769a06b92 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:06.185621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d5a63-42c3-47ef-bffc-c21769a06b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:06.199358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d5a63-42c3-47ef-bffc-c21769a06b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:06.201274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d5a63-42c3-47ef-bffc-c21769a06b92 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-9e1d5a63-42c3-47ef-bffc-c21769a06b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:06.219261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:06.220302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:06.221197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:06.221197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:06.221197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:06.229646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 480/954] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:06.235860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4d7e7-eb81-4d23-b331-937048b9e415 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:06.312554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:06.346034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e4d7e7-eb81-4d23-b331-937048b9e415 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:06.389449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea51f0-9bcb-4c96-aa2a-db69fa4ca6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:06.390118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea51f0-9bcb-4c96-aa2a-db69fa4ca6bd None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:06.390394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea51f0-9bcb-4c96-aa2a-db69fa4ca6bd None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:06.390667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea51f0-9bcb-4c96-aa2a-db69fa4ca6bd None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:06.391330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea51f0-9bcb-4c96-aa2a-db69fa4ca6bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:06.391628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea51f0-9bcb-4c96-aa2a-db69fa4ca6bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:06.392994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea51f0-9bcb-4c96-aa2a-db69fa4ca6bd None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:06.408346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 475/955] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:06.415071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d5a63-42c3-47ef-bffc-c21769a06b92 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:06.431009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2ab6f-5126-4d27-8403-5da60cb1fc5b None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:06.431308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2ab6f-5126-4d27-8403-5da60cb1fc5b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:06.431666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2ab6f-5126-4d27-8403-5da60cb1fc5b None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:06.433284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 476/956] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:06.438692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907177a-3b2c-4853-855b-cf35773d2ef9 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:06.438970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907177a-3b2c-4853-855b-cf35773d2ef9 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:06.439239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907177a-3b2c-4853-855b-cf35773d2ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:06.459068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-0907177a-3b2c-4853-855b-cf35773d2ef9 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:06.461147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0907177a-3b2c-4853-855b-cf35773d2ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:06.546864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d5a63-42c3-47ef-bffc-c21769a06b92 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:06.554675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 477/957] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:06.556035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bbef3-5668-4356-b45e-643d9c8876c5 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:06.556324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bbef3-5668-4356-b45e-643d9c8876c5 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:06.556597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bbef3-5668-4356-b45e-643d9c8876c5 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:06.561999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 478/958] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:06.565645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8375916-b1ad-402e-bc72-21eb8d7392e7 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:06.566120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8375916-b1ad-402e-bc72-21eb8d7392e7 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:06.566560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8375916-b1ad-402e-bc72-21eb8d7392e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:06.588229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-b8375916-b1ad-402e-bc72-21eb8d7392e7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:06.589802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b8375916-b1ad-402e-bc72-21eb8d7392e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:06.611854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f59ac-044b-4b20-bb75-9eb7ae6015d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:06.612530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f59ac-044b-4b20-bb75-9eb7ae6015d5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:06.612784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f59ac-044b-4b20-bb75-9eb7ae6015d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:06.613029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f59ac-044b-4b20-bb75-9eb7ae6015d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:06.622102 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f59ac-044b-4b20-bb75-9eb7ae6015d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:06.623537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f59ac-044b-4b20-bb75-9eb7ae6015d5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-f39f59ac-044b-4b20-bb75-9eb7ae6015d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:33:06.636416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:06.637530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 481/959] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Aug 9 16:33:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:06.643656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f59ac-044b-4b20-bb75-9eb7ae6015d5 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:06.664671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:06.675361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:06.676403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 479/960] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:06.677289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8375916-b1ad-402e-bc72-21eb8d7392e7 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:06.763030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f59ac-044b-4b20-bb75-9eb7ae6015d5 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:06.803548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885fbfb-9400-4053-85e1-b6ee2ad95939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:06.804739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885fbfb-9400-4053-85e1-b6ee2ad95939 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:06.804936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885fbfb-9400-4053-85e1-b6ee2ad95939 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:06.805193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885fbfb-9400-4053-85e1-b6ee2ad95939 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:06.805764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885fbfb-9400-4053-85e1-b6ee2ad95939 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:06.807258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885fbfb-9400-4053-85e1-b6ee2ad95939 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:06.809038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8375916-b1ad-402e-bc72-21eb8d7392e7 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-9885fbfb-9400-4053-85e1-b6ee2ad95939 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:06.817331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:06.818453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 16:33:06.819482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:06.824040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 482/961] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:33:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:06.824817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885fbfb-9400-4053-85e1-b6ee2ad95939 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:06.850406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c66be-f474-4777-b02b-6468e3f2489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:06.851040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c66be-f474-4777-b02b-6468e3f2489f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:06.851478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c66be-f474-4777-b02b-6468e3f2489f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:06.851924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c66be-f474-4777-b02b-6468e3f2489f None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:06.853201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c66be-f474-4777-b02b-6468e3f2489f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:06.853669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c66be-f474-4777-b02b-6468e3f2489f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:06.855387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942c66be-f474-4777-b02b-6468e3f2489f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:06.877904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 480/962] 10.222.0.79 () {56 vars in 1149 bytes} [Tue Aug 9 16:33:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:06.879007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c66be-f474-4777-b02b-6468e3f2489f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:06.943543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:06.951992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:06.964871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885fbfb-9400-4053-85e1-b6ee2ad95939 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:06.999896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942c66be-f474-4777-b02b-6468e3f2489f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:07.010187 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37857b08-4cfc-4046-b428-149c63e50d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:07.010900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37857b08-4cfc-4046-b428-149c63e50d46 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:07.011246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37857b08-4cfc-4046-b428-149c63e50d46 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:07.011573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37857b08-4cfc-4046-b428-149c63e50d46 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:07.012218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37857b08-4cfc-4046-b428-149c63e50d46 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:07.014525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37857b08-4cfc-4046-b428-149c63e50d46 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-37857b08-4cfc-4046-b428-149c63e50d46 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:07.029580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:07.036544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:07.037654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 483/963] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:33:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:07.039248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37857b08-4cfc-4046-b428-149c63e50d46 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:07.056196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec6243-2193-4806-9b35-30fc0df64ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:07.056793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6243-2193-4806-9b35-30fc0df64ff5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:07.057061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6243-2193-4806-9b35-30fc0df64ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:07.057387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec6243-2193-4806-9b35-30fc0df64ff5 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:07.057941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec6243-2193-4806-9b35-30fc0df64ff5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:07.058261 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec6243-2193-4806-9b35-30fc0df64ff5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:07.059610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec6243-2193-4806-9b35-30fc0df64ff5 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:07.065394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 481/964] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:33:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:07.066410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec6243-2193-4806-9b35-30fc0df64ff5 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:07.168934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37857b08-4cfc-4046-b428-149c63e50d46 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:07.183153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec6243-2193-4806-9b35-30fc0df64ff5 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:07.199050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:07.220984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fd412-b9c4-4b5c-81d7-b73bd7e4996a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:07.221666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fd412-b9c4-4b5c-81d7-b73bd7e4996a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:07.221974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fd412-b9c4-4b5c-81d7-b73bd7e4996a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:07.222246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fd412-b9c4-4b5c-81d7-b73bd7e4996a None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:07.222876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062fd412-b9c4-4b5c-81d7-b73bd7e4996a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:07.223195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062fd412-b9c4-4b5c-81d7-b73bd7e4996a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:07.223795 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c08285-79f7-486c-85e2-fe87fc61b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:07.224449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c08285-79f7-486c-85e2-fe87fc61b14d None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:07.224641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062fd412-b9c4-4b5c-81d7-b73bd7e4996a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:07.224911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c08285-79f7-486c-85e2-fe87fc61b14d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:07.225281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c08285-79f7-486c-85e2-fe87fc61b14d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:07.237283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 482/965] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Aug 9 16:33:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:07.238024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fd412-b9c4-4b5c-81d7-b73bd7e4996a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:07.254356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:07.308477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c08285-79f7-486c-85e2-fe87fc61b14d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:07.310690 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c08285-79f7-486c-85e2-fe87fc61b14d None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:07.360992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fd412-b9c4-4b5c-81d7-b73bd7e4996a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:07.402260 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3502f71-9b3d-4dba-89d2-b245d5064c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:07.402870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3502f71-9b3d-4dba-89d2-b245d5064c34 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:07.403144 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3502f71-9b3d-4dba-89d2-b245d5064c34 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:07.403406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3502f71-9b3d-4dba-89d2-b245d5064c34 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:07.404043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3502f71-9b3d-4dba-89d2-b245d5064c34 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:07.404338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3502f71-9b3d-4dba-89d2-b245d5064c34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:07.405712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3502f71-9b3d-4dba-89d2-b245d5064c34 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:07.410636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 483/966] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:33:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:07.411448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3502f71-9b3d-4dba-89d2-b245d5064c34 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:07.475800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 484/967] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:07 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:33:07.481488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c08285-79f7-486c-85e2-fe87fc61b14d None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:07.484775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:07.568358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3502f71-9b3d-4dba-89d2-b245d5064c34 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:07.594162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:07.610335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b9a7f-5045-445d-9c01-e3bc0466ad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:07.610944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b9a7f-5045-445d-9c01-e3bc0466ad2d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:07.611221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b9a7f-5045-445d-9c01-e3bc0466ad2d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:07.611691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b9a7f-5045-445d-9c01-e3bc0466ad2d None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:07.612178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b9a7f-5045-445d-9c01-e3bc0466ad2d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:07.612479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b9a7f-5045-445d-9c01-e3bc0466ad2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:07.613863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b9a7f-5045-445d-9c01-e3bc0466ad2d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:07.621480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 484/968] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Aug 9 16:33:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:07.629933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b9a7f-5045-445d-9c01-e3bc0466ad2d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:07.650008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c08285-79f7-486c-85e2-fe87fc61b14d None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:07.692578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b273f4a-82df-4650-b15e-3bd4740e5986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:07.693577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b273f4a-82df-4650-b15e-3bd4740e5986 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:07.694181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b273f4a-82df-4650-b15e-3bd4740e5986 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:07.694672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b273f4a-82df-4650-b15e-3bd4740e5986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:07.707080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b273f4a-82df-4650-b15e-3bd4740e5986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:07.709010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b273f4a-82df-4650-b15e-3bd4740e5986 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-3b273f4a-82df-4650-b15e-3bd4740e5986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:07.726191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:07.727151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:07.728152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:07.728152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:07.728152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:07.730407 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 485/969] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:07.735605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b273f4a-82df-4650-b15e-3bd4740e5986 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:07.779281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b9a7f-5045-445d-9c01-e3bc0466ad2d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:07.788224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:07.817840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c02640-e4f1-4c02-b13a-f5483bb069b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:07.818434 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c02640-e4f1-4c02-b13a-f5483bb069b8 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:07.818691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c02640-e4f1-4c02-b13a-f5483bb069b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:07.818961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c02640-e4f1-4c02-b13a-f5483bb069b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:07.819669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c02640-e4f1-4c02-b13a-f5483bb069b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:07.821687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c02640-e4f1-4c02-b13a-f5483bb069b8 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:07.873837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b273f4a-82df-4650-b15e-3bd4740e5986 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:07.882414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:07.922484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 485/970] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:33:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:33:07.932771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe893a7-5eb8-48fb-a382-0cc64ca7a8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:07.933393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe893a7-5eb8-48fb-a382-0cc64ca7a8d5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:07.933666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe893a7-5eb8-48fb-a382-0cc64ca7a8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:07.940859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe893a7-5eb8-48fb-a382-0cc64ca7a8d5 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:07.940859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe893a7-5eb8-48fb-a382-0cc64ca7a8d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:07.940859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe893a7-5eb8-48fb-a382-0cc64ca7a8d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:07.940859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe893a7-5eb8-48fb-a382-0cc64ca7a8d5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:07.952097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 486/971] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:07.957552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c02640-e4f1-4c02-b13a-f5483bb069b8 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:08.101472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c02640-e4f1-4c02-b13a-f5483bb069b8 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:08.149895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7708b95-8691-4d97-9cbe-a14f7f72b77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:08.150980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7708b95-8691-4d97-9cbe-a14f7f72b77c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:08.151521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7708b95-8691-4d97-9cbe-a14f7f72b77c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:08.152018 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7708b95-8691-4d97-9cbe-a14f7f72b77c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:08.160165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7708b95-8691-4d97-9cbe-a14f7f72b77c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:08.161989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7708b95-8691-4d97-9cbe-a14f7f72b77c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-c7708b95-8691-4d97-9cbe-a14f7f72b77c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:33:08.175682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:08.176722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:08.177612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:33:08.177612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:33:08.177612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:33:08.177612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:33:08.177612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:33:08.177612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:33:08.177612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:08.177612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:08.180653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 486/972] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:33:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:08.186424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe893a7-5eb8-48fb-a382-0cc64ca7a8d5 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:08.308639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe893a7-5eb8-48fb-a382-0cc64ca7a8d5 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:08.347928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ac5f0-7596-4f46-8bcb-7baf47ff548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:08.348934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ac5f0-7596-4f46-8bcb-7baf47ff548c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:08.349417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ac5f0-7596-4f46-8bcb-7baf47ff548c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:08.349885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ac5f0-7596-4f46-8bcb-7baf47ff548c None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:08.350903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ac5f0-7596-4f46-8bcb-7baf47ff548c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:08.352029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ac5f0-7596-4f46-8bcb-7baf47ff548c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:08.353671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ac5f0-7596-4f46-8bcb-7baf47ff548c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:08.367375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 487/973] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:08.376571 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7708b95-8691-4d97-9cbe-a14f7f72b77c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:08.404210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:08.465319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4d154-3c9b-46ef-bb8a-eebec3ac4ad1 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:08.465657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4d154-3c9b-46ef-bb8a-eebec3ac4ad1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:08.465878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4d154-3c9b-46ef-bb8a-eebec3ac4ad1 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:08.467647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 488/974] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:08.477233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781615dc-6f3f-4ff2-bc21-5e51d3ceab9f None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:08.477867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781615dc-6f3f-4ff2-bc21-5e51d3ceab9f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:08.477867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781615dc-6f3f-4ff2-bc21-5e51d3ceab9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:08.497295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7708b95-8691-4d97-9cbe-a14f7f72b77c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:08.513623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-781615dc-6f3f-4ff2-bc21-5e51d3ceab9f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:08.517779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-781615dc-6f3f-4ff2-bc21-5e51d3ceab9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:08.550533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1fac5-4e7a-4509-8465-61a3f905f794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:08.551482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1fac5-4e7a-4509-8465-61a3f905f794 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:08.552010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1fac5-4e7a-4509-8465-61a3f905f794 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:08.552486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1fac5-4e7a-4509-8465-61a3f905f794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:08.553721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1fac5-4e7a-4509-8465-61a3f905f794 None admin] RBAC: Adding query filter params (role.id=f4cfecbeb0f44d2aa4a275e60038fa7c, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=a010ea3529644fca951a7eef28dbe97e) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:08.554371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1fac5-4e7a-4509-8465-61a3f905f794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:08.556005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1fac5-4e7a-4509-8465-61a3f905f794 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:08.564213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 487/975] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:33:08 2022] GET /identity/v3/role_assignments?user.id=a010ea3529644fca951a7eef28dbe97e&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=f4cfecbeb0f44d2aa4a275e60038fa7c => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:08.566321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:08.634296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 489/976] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:08.662672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1fac5-4e7a-4509-8465-61a3f905f794 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:08.797939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:08.818022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1fac5-4e7a-4509-8465-61a3f905f794 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:08.864310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffa0a0-eeb6-41c2-8391-59004ec38486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:08.864922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffa0a0-eeb6-41c2-8391-59004ec38486 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:08.865449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffa0a0-eeb6-41c2-8391-59004ec38486 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:08.865829 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffa0a0-eeb6-41c2-8391-59004ec38486 None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:08.866283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffa0a0-eeb6-41c2-8391-59004ec38486 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:08.868229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffa0a0-eeb6-41c2-8391-59004ec38486 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:08.928134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.sql.core [None req-06ffa0a0-eeb6-41c2-8391-59004ec38486 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 16:33:08.928134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 16:33:08.928134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [parameters: {'user_id': '6d32e1fba55c474d8cea7b74d4a6333b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 16:33:08.928134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86843) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-06ffa0a0-eeb6-41c2-8391-59004ec38486 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application conn.query(q) Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application result.read() Aug 09 16:33:08.937342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 16:33:08.940064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application next(self.gen) Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application yield resource Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application next(self.gen) Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self.session.rollback() Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self.force_reraise() Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise self.value Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application session.commit() Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 09 16:33:08.941099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._prepare_impl() Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self.session.flush() Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._flush(objects) Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application compat.raise_( Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application flush_context.execute() Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rec.execute(self) Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 09 16:33:08.942081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 09 16:33:08.942978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application result = self._query(query) Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application conn.query(q) Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application result.read() Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 09 16:33:08.943838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [parameters: {'user_id': '6d32e1fba55c474d8cea7b74d4a6333b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:08.944655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 09 16:33:08.945496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 09 16:33:08.947366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 09 16:33:08.947366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 09 16:33:08.947366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 09 16:33:08.947366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 09 16:33:08.947366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 09 16:33:08.947366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:08.947366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 488/977] 10.222.0.79 () {60 vars in 1131 bytes} [Tue Aug 9 16:33:08 2022] POST /identity/v3/users => generated 160 bytes in 277 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:08.948857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781615dc-6f3f-4ff2-bc21-5e51d3ceab9f None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:09.061177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781615dc-6f3f-4ff2-bc21-5e51d3ceab9f None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:09.099211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984059a-ee0f-4949-bd35-d15b9571a431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:09.099843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984059a-ee0f-4949-bd35-d15b9571a431 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:09.100112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984059a-ee0f-4949-bd35-d15b9571a431 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:09.100375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984059a-ee0f-4949-bd35-d15b9571a431 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:09.109332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984059a-ee0f-4949-bd35-d15b9571a431 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:09.110891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984059a-ee0f-4949-bd35-d15b9571a431 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-1984059a-ee0f-4949-bd35-d15b9571a431 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:33:09.124300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:09.126587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:09.128830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:09.128830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:09.128830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:33:09.128830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:33:09.128830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:33:09.128830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:33:09.128830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:33:09.128830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:33:09.128830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 09 16:33:09.128830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:09.130217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 490/978] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:33:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:09.134101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffa0a0-eeb6-41c2-8391-59004ec38486 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:09.279126 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffa0a0-eeb6-41c2-8391-59004ec38486 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:09.321769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed03ed-98f3-4210-8b78-a26678f45d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:09.322403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed03ed-98f3-4210-8b78-a26678f45d3c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:09.322681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed03ed-98f3-4210-8b78-a26678f45d3c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:09.323125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed03ed-98f3-4210-8b78-a26678f45d3c None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:09.323911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed03ed-98f3-4210-8b78-a26678f45d3c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:09.324211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed03ed-98f3-4210-8b78-a26678f45d3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:09.325885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed03ed-98f3-4210-8b78-a26678f45d3c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:09.373493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 489/979] 10.222.0.79 () {56 vars in 1101 bytes} [Tue Aug 9 16:33:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:09.815569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a6929-a9c0-4b8b-8b16-2a904f753110 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:09.816217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a6929-a9c0-4b8b-8b16-2a904f753110 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:09.816217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a6929-a9c0-4b8b-8b16-2a904f753110 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:09.818606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 491/980] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:09.824001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55517a42-8bbc-4896-a33d-37dab0be95b6 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:09.824541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55517a42-8bbc-4896-a33d-37dab0be95b6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:09.824541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55517a42-8bbc-4896-a33d-37dab0be95b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:09.838586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:09.847556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-55517a42-8bbc-4896-a33d-37dab0be95b6 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:09.849102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-55517a42-8bbc-4896-a33d-37dab0be95b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:09.943549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 490/981] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:09.968355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a6929-a9c0-4b8b-8b16-2a904f753110 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:09.998769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:10.130456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a6929-a9c0-4b8b-8b16-2a904f753110 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:10.167454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1883f9-46d1-47b6-bd33-1c0b84251ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:10.168049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1883f9-46d1-47b6-bd33-1c0b84251ee0 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:10.168321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1883f9-46d1-47b6-bd33-1c0b84251ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:10.168584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1883f9-46d1-47b6-bd33-1c0b84251ee0 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:10.169248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1883f9-46d1-47b6-bd33-1c0b84251ee0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:10.170638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1883f9-46d1-47b6-bd33-1c0b84251ee0 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:10.245993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 492/982] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:33:09 2022] POST /identity/v3/services => generated 242 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:33:10.288028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ad0e6-397b-4f87-990d-d90fc8866920 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:10.288346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ad0e6-397b-4f87-990d-d90fc8866920 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:10.289096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ad0e6-397b-4f87-990d-d90fc8866920 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:10.290776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 491/983] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:10.296922 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740397cd-79cb-4b60-b50f-2a8c9078e0f5 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:10.297260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740397cd-79cb-4b60-b50f-2a8c9078e0f5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:10.297507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740397cd-79cb-4b60-b50f-2a8c9078e0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:10.318274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-740397cd-79cb-4b60-b50f-2a8c9078e0f5 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:10.319552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-740397cd-79cb-4b60-b50f-2a8c9078e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:10.400927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:10.416650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 493/984] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:10.440379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ad0e6-397b-4f87-990d-d90fc8866920 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:10.564608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ad0e6-397b-4f87-990d-d90fc8866920 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:10.602373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a40d4-996e-4e7f-aee5-9714cb2f6f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:10.602998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a40d4-996e-4e7f-aee5-9714cb2f6f8f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:10.603269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a40d4-996e-4e7f-aee5-9714cb2f6f8f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:10.603530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a40d4-996e-4e7f-aee5-9714cb2f6f8f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:10.604193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a40d4-996e-4e7f-aee5-9714cb2f6f8f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:10.605620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a40d4-996e-4e7f-aee5-9714cb2f6f8f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:10.608564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-457a40d4-996e-4e7f-aee5-9714cb2f6f8f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:10.615750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:10.632676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:10.634310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 492/985] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:33:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:10.635738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740397cd-79cb-4b60-b50f-2a8c9078e0f5 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:10.759865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740397cd-79cb-4b60-b50f-2a8c9078e0f5 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:10.799182 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870a066-fe0a-4aa1-b1ee-d4eec665873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:10.799793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870a066-fe0a-4aa1-b1ee-d4eec665873b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:10.800062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870a066-fe0a-4aa1-b1ee-d4eec665873b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:10.800324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870a066-fe0a-4aa1-b1ee-d4eec665873b None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:10.800948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870a066-fe0a-4aa1-b1ee-d4eec665873b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:10.801243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870a066-fe0a-4aa1-b1ee-d4eec665873b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:10.802612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870a066-fe0a-4aa1-b1ee-d4eec665873b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:10.807896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 494/986] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Aug 9 16:33:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:10.813774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a40d4-996e-4e7f-aee5-9714cb2f6f8f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:10.931141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a40d4-996e-4e7f-aee5-9714cb2f6f8f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:10.966556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa003c1f-5ff0-4319-bd0f-a59c3463cd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:10.967416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa003c1f-5ff0-4319-bd0f-a59c3463cd73 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:10.967688 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa003c1f-5ff0-4319-bd0f-a59c3463cd73 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:10.967949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa003c1f-5ff0-4319-bd0f-a59c3463cd73 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:10.968570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa003c1f-5ff0-4319-bd0f-a59c3463cd73 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:10.968865 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa003c1f-5ff0-4319-bd0f-a59c3463cd73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:10.970247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa003c1f-5ff0-4319-bd0f-a59c3463cd73 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:10.975395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 493/987] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Aug 9 16:33:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:10.981230 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870a066-fe0a-4aa1-b1ee-d4eec665873b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:11.024804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26e8b-6fb6-4749-ab6b-685a95984f31 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:11.025181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26e8b-6fb6-4749-ab6b-685a95984f31 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:11.025663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b26e8b-6fb6-4749-ab6b-685a95984f31 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:11.030015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 494/988] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:11.034709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615e21d-36db-4f16-b8b3-ac2dff92384f None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:11.035433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615e21d-36db-4f16-b8b3-ac2dff92384f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:11.036105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615e21d-36db-4f16-b8b3-ac2dff92384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:11.046657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:11.065294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9615e21d-36db-4f16-b8b3-ac2dff92384f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:11.067258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9615e21d-36db-4f16-b8b3-ac2dff92384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:11.087324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870a066-fe0a-4aa1-b1ee-d4eec665873b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:11.135424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caca54-98b9-4771-a8a8-7021b157c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:11.136035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caca54-98b9-4771-a8a8-7021b157c005 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:11.136327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caca54-98b9-4771-a8a8-7021b157c005 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:11.136582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caca54-98b9-4771-a8a8-7021b157c005 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:11.137238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49caca54-98b9-4771-a8a8-7021b157c005 None admin] RBAC: Adding query filter params (interface=admin, service_id=f4f7e1bf31ff404287c39408a211dd64, region_id=RegionOne) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:11.137543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49caca54-98b9-4771-a8a8-7021b157c005 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:11.138927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49caca54-98b9-4771-a8a8-7021b157c005 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:11.144605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 495/989] 10.222.0.79 () {56 vars in 1249 bytes} [Tue Aug 9 16:33:10 2022] GET /identity/v3/endpoints?service_id=f4f7e1bf31ff404287c39408a211dd64&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:11.150507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:11.151803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caca54-98b9-4771-a8a8-7021b157c005 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:11.161292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 495/990] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:11.186108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615e21d-36db-4f16-b8b3-ac2dff92384f None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:11.265804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caca54-98b9-4771-a8a8-7021b157c005 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:11.294027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:11.302937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ed8db-2716-4a6c-a69f-e46c1bac88dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:11.303573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ed8db-2716-4a6c-a69f-e46c1bac88dc None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:11.309734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ed8db-2716-4a6c-a69f-e46c1bac88dc None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:11.309734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ed8db-2716-4a6c-a69f-e46c1bac88dc None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:11.309734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ed8db-2716-4a6c-a69f-e46c1bac88dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:11.309734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ed8db-2716-4a6c-a69f-e46c1bac88dc None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:11.325989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 496/991] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Aug 9 16:33:11 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:33:11.326347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615e21d-36db-4f16-b8b3-ac2dff92384f None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:11.369038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e45f6-edcd-4afb-9049-5b1b73008cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:11.369685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e45f6-edcd-4afb-9049-5b1b73008cbd None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:11.369972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e45f6-edcd-4afb-9049-5b1b73008cbd None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:11.370258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e45f6-edcd-4afb-9049-5b1b73008cbd None admin] PATH_INFO: `/v3/services/image` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:11.370863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e45f6-edcd-4afb-9049-5b1b73008cbd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:11.372291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e45f6-edcd-4afb-9049-5b1b73008cbd None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-343e45f6-edcd-4afb-9049-5b1b73008cbd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:11.382744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:11.383939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 16:33:11.384982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:11.386719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 496/992] 10.222.0.79 () {56 vars in 1105 bytes} [Tue Aug 9 16:33:11 2022] GET /identity/v3/services/image => generated 86 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:11.390245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ed8db-2716-4a6c-a69f-e46c1bac88dc None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:11.547008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ed8db-2716-4a6c-a69f-e46c1bac88dc None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:11.603215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae70e4-a023-45b8-919e-9c8b174001b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:11.604459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae70e4-a023-45b8-919e-9c8b174001b4 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:11.605057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae70e4-a023-45b8-919e-9c8b174001b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:11.605657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae70e4-a023-45b8-919e-9c8b174001b4 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:11.606959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae70e4-a023-45b8-919e-9c8b174001b4 None admin] RBAC: Adding query filter params (name=image) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:11.607679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae70e4-a023-45b8-919e-9c8b174001b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:11.610057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae70e4-a023-45b8-919e-9c8b174001b4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:11.617094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 497/993] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:11.625722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e45f6-edcd-4afb-9049-5b1b73008cbd None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:11.716069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:11.768619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e45f6-edcd-4afb-9049-5b1b73008cbd None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:11.805147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b936bfe-3afe-45ab-940e-f4b35287b730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:11.805657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b936bfe-3afe-45ab-940e-f4b35287b730 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:11.805930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b936bfe-3afe-45ab-940e-f4b35287b730 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:11.806222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b936bfe-3afe-45ab-940e-f4b35287b730 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:11.806845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b936bfe-3afe-45ab-940e-f4b35287b730 None admin] RBAC: Adding query filter params (type=image) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:11.807140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b936bfe-3afe-45ab-940e-f4b35287b730 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:11.808498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b936bfe-3afe-45ab-940e-f4b35287b730 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:11.814505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 497/994] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:12.459835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c8a19-0e3f-4739-8bae-35ceab1ee9c3 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:12.460402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c8a19-0e3f-4739-8bae-35ceab1ee9c3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:12.460663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c8a19-0e3f-4739-8bae-35ceab1ee9c3 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:12.462217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 498/995] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:12.468087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e04b86-2046-4906-aef8-19b1573d47c4 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:12.468547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e04b86-2046-4906-aef8-19b1573d47c4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:12.468622 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e04b86-2046-4906-aef8-19b1573d47c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:12.487984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-21e04b86-2046-4906-aef8-19b1573d47c4 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:12.489207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-21e04b86-2046-4906-aef8-19b1573d47c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:12.565989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 498/996] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:12.588673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c8a19-0e3f-4739-8bae-35ceab1ee9c3 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:12.670695 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:12.742549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c8a19-0e3f-4739-8bae-35ceab1ee9c3 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:12.793300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc0021-55d2-40ee-b414-ab5b572fa32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:12.794037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc0021-55d2-40ee-b414-ab5b572fa32e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:12.794319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc0021-55d2-40ee-b414-ab5b572fa32e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:12.794592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc0021-55d2-40ee-b414-ab5b572fa32e None admin] PATH_INFO: `/v3/services/compute` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:12.795203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc0021-55d2-40ee-b414-ab5b572fa32e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:12.796735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dc0021-55d2-40ee-b414-ab5b572fa32e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-b3dc0021-55d2-40ee-b414-ab5b572fa32e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:12.809850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:12.817337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:12.818533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 499/997] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:33:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:12.820957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e04b86-2046-4906-aef8-19b1573d47c4 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:12.887836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:12.936709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e04b86-2046-4906-aef8-19b1573d47c4 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:12.981115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fba3d2-0576-4521-a517-77901ddc1158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:12.981725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba3d2-0576-4521-a517-77901ddc1158 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:12.982026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba3d2-0576-4521-a517-77901ddc1158 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:12.982298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fba3d2-0576-4521-a517-77901ddc1158 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:12.982933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fba3d2-0576-4521-a517-77901ddc1158 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:12.983231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fba3d2-0576-4521-a517-77901ddc1158 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:12.984592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fba3d2-0576-4521-a517-77901ddc1158 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:12.992647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 499/998] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:33:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:12.998510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc0021-55d2-40ee-b414-ab5b572fa32e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:13.142326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc0021-55d2-40ee-b414-ab5b572fa32e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:13.162371 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:13.180966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4226ce5-2a07-4c76-bdaa-2bb2a545d22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:13.181566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4226ce5-2a07-4c76-bdaa-2bb2a545d22f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:13.181843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4226ce5-2a07-4c76-bdaa-2bb2a545d22f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:13.182138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4226ce5-2a07-4c76-bdaa-2bb2a545d22f None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:13.182773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4226ce5-2a07-4c76-bdaa-2bb2a545d22f None admin] RBAC: Adding query filter params (type=compute) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:13.183128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4226ce5-2a07-4c76-bdaa-2bb2a545d22f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:13.184533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4226ce5-2a07-4c76-bdaa-2bb2a545d22f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:13.190207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 500/999] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:33:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:13.196548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fba3d2-0576-4521-a517-77901ddc1158 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:13.303980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fba3d2-0576-4521-a517-77901ddc1158 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:13.323249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:13.343538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3b5770-bd59-4c25-bdb2-634914d4cbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:13.350953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b5770-bd59-4c25-bdb2-634914d4cbc3 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:13.350953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b5770-bd59-4c25-bdb2-634914d4cbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:13.350953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3b5770-bd59-4c25-bdb2-634914d4cbc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:13.350953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b5770-bd59-4c25-bdb2-634914d4cbc3 None admin] RBAC: Adding query filter params (interface=public, service_id=c44886fc86c14fda80c1e87d176d76f5, region_id=RegionOne) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:13.350953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b5770-bd59-4c25-bdb2-634914d4cbc3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:13.350953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3b5770-bd59-4c25-bdb2-634914d4cbc3 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:13.353366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 500/1000] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Aug 9 16:33:13 2022] GET /identity/v3/endpoints?service_id=c44886fc86c14fda80c1e87d176d76f5&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:13.359555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4226ce5-2a07-4c76-bdaa-2bb2a545d22f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:13.535076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4226ce5-2a07-4c76-bdaa-2bb2a545d22f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:13.572818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9fccc-27bf-4ee6-ad05-f24f203480a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:13.573341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9fccc-27bf-4ee6-ad05-f24f203480a9 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:13.573618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9fccc-27bf-4ee6-ad05-f24f203480a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:13.573883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9fccc-27bf-4ee6-ad05-f24f203480a9 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:13.574516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9fccc-27bf-4ee6-ad05-f24f203480a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:13.576071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9fccc-27bf-4ee6-ad05-f24f203480a9 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:13.589990 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 501/1001] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Aug 9 16:33:13 2022] GET /identity/v3/services => generated 1727 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:33:13.602256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb88e3-298f-439c-8af1-e4dfc6a6db78 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:13.602880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb88e3-298f-439c-8af1-e4dfc6a6db78 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:13.603033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb88e3-298f-439c-8af1-e4dfc6a6db78 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:13.604543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 501/1002] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:13.610824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292234-6bf9-4aa9-ab88-36feef65d2e3 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:13.611101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292234-6bf9-4aa9-ab88-36feef65d2e3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:13.611361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292234-6bf9-4aa9-ab88-36feef65d2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:13.631982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d6292234-6bf9-4aa9-ab88-36feef65d2e3 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:13.633224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d6292234-6bf9-4aa9-ab88-36feef65d2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:13.693917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:13.729589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 502/1003] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:13.754104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb88e3-298f-439c-8af1-e4dfc6a6db78 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:13.874707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb88e3-298f-439c-8af1-e4dfc6a6db78 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:13.911752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92b15c-9391-48b7-80f6-c7b897e93b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:13.912375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92b15c-9391-48b7-80f6-c7b897e93b87 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:13.912652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92b15c-9391-48b7-80f6-c7b897e93b87 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:13.912921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92b15c-9391-48b7-80f6-c7b897e93b87 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:13.914073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92b15c-9391-48b7-80f6-c7b897e93b87 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:13.915507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92b15c-9391-48b7-80f6-c7b897e93b87 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-4a92b15c-9391-48b7-80f6-c7b897e93b87 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:13.925641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:13.926746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 09 16:33:13.928453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:13.930250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 502/1004] 10.222.0.79 () {56 vars in 1126 bytes} [Tue Aug 9 16:33:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:13.934296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292234-6bf9-4aa9-ab88-36feef65d2e3 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:13.997656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210b95e-816d-4caa-ae7c-84ba7e6b79e6 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:13.997927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210b95e-816d-4caa-ae7c-84ba7e6b79e6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:13.998346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210b95e-816d-4caa-ae7c-84ba7e6b79e6 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:14.003901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 503/1005] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:14.005146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2469084-a0f2-484d-a41e-b08e86a9ce04 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:14.005429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2469084-a0f2-484d-a41e-b08e86a9ce04 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:14.005695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2469084-a0f2-484d-a41e-b08e86a9ce04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:14.027288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-b2469084-a0f2-484d-a41e-b08e86a9ce04 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:14.029274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b2469084-a0f2-484d-a41e-b08e86a9ce04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:14.063946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292234-6bf9-4aa9-ab88-36feef65d2e3 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:14.085760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:14.124353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d69da0-e326-43f4-9c54-6decbd19c972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:14.125811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d69da0-e326-43f4-9c54-6decbd19c972 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:14.126335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d69da0-e326-43f4-9c54-6decbd19c972 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:14.126782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d69da0-e326-43f4-9c54-6decbd19c972 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:14.127600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d69da0-e326-43f4-9c54-6decbd19c972 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:14.128079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d69da0-e326-43f4-9c54-6decbd19c972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:14.129770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d69da0-e326-43f4-9c54-6decbd19c972 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:14.130042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 504/1006] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:14.141636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 503/1007] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Aug 9 16:33:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:14.143249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2469084-a0f2-484d-a41e-b08e86a9ce04 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:14.148766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d69da0-e326-43f4-9c54-6decbd19c972 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:14.229719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:14.284975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2469084-a0f2-484d-a41e-b08e86a9ce04 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:14.307583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d69da0-e326-43f4-9c54-6decbd19c972 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:14.338294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf15c09-bdd7-4649-ae61-f79ce0938324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:14.339284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf15c09-bdd7-4649-ae61-f79ce0938324 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:14.339801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf15c09-bdd7-4649-ae61-f79ce0938324 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:14.340267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf15c09-bdd7-4649-ae61-f79ce0938324 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:14.341667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf15c09-bdd7-4649-ae61-f79ce0938324 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:14.342796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf15c09-bdd7-4649-ae61-f79ce0938324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:14.344754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf15c09-bdd7-4649-ae61-f79ce0938324 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:14.352742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 505/1008] 10.222.0.79 () {56 vars in 1122 bytes} [Tue Aug 9 16:33:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:14.360492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf15c09-bdd7-4649-ae61-f79ce0938324 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:14.371291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281d071a-f329-4813-8588-7ac0a4f220af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:14.371914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d071a-f329-4813-8588-7ac0a4f220af None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:14.372215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d071a-f329-4813-8588-7ac0a4f220af None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:14.372505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281d071a-f329-4813-8588-7ac0a4f220af None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:14.373176 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d071a-f329-4813-8588-7ac0a4f220af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:14.374600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281d071a-f329-4813-8588-7ac0a4f220af None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:14.428460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:14.494095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf15c09-bdd7-4649-ae61-f79ce0938324 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:14.506255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 504/1009] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:33:14 2022] POST /identity/v3/services => generated 242 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:33:14.565277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574925e-5aa2-48e0-a496-f1ed64e1e449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:14.566094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574925e-5aa2-48e0-a496-f1ed64e1e449 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:14.566451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574925e-5aa2-48e0-a496-f1ed64e1e449 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:14.567118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574925e-5aa2-48e0-a496-f1ed64e1e449 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:14.567433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574925e-5aa2-48e0-a496-f1ed64e1e449 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:14.568863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574925e-5aa2-48e0-a496-f1ed64e1e449 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:14.655709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 506/1010] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:33:14 2022] POST /identity/v3/endpoints => generated 325 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:33:15.731752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126af9f-d104-4a92-a792-6e51e8c759f6 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:15.732331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126af9f-d104-4a92-a792-6e51e8c759f6 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:15.732331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126af9f-d104-4a92-a792-6e51e8c759f6 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:15.733884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 505/1011] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:15.739639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17db8a9-4aeb-47fe-8ec7-fcfce40a286c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:15.740147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17db8a9-4aeb-47fe-8ec7-fcfce40a286c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:15.740224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17db8a9-4aeb-47fe-8ec7-fcfce40a286c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:15.758643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-e17db8a9-4aeb-47fe-8ec7-fcfce40a286c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:15.759847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e17db8a9-4aeb-47fe-8ec7-fcfce40a286c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:15.850524 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 507/1012] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:15 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:15.874126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126af9f-d104-4a92-a792-6e51e8c759f6 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:15.987473 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:15.992560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126af9f-d104-4a92-a792-6e51e8c759f6 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:16.030000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e562f-1f68-49a5-9a5f-78c0b7282f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:16.031019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e562f-1f68-49a5-9a5f-78c0b7282f06 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:16.031518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e562f-1f68-49a5-9a5f-78c0b7282f06 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:16.031916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e562f-1f68-49a5-9a5f-78c0b7282f06 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:16.032632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e562f-1f68-49a5-9a5f-78c0b7282f06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:16.034455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e562f-1f68-49a5-9a5f-78c0b7282f06 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-529e562f-1f68-49a5-9a5f-78c0b7282f06 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:16.044520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:16.046035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 09 16:33:16.046928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:16.054003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 506/1013] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:33:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:16.054792 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17db8a9-4aeb-47fe-8ec7-fcfce40a286c None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:16.086655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:16.181396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17db8a9-4aeb-47fe-8ec7-fcfce40a286c None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:16.226770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac8cd5-4112-49cc-82e9-c90c8a75a023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:16.227382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac8cd5-4112-49cc-82e9-c90c8a75a023 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:16.227659 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac8cd5-4112-49cc-82e9-c90c8a75a023 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:16.227928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac8cd5-4112-49cc-82e9-c90c8a75a023 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:16.228560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac8cd5-4112-49cc-82e9-c90c8a75a023 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:16.228858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac8cd5-4112-49cc-82e9-c90c8a75a023 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:16.230289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac8cd5-4112-49cc-82e9-c90c8a75a023 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:16.237367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 508/1014] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:33:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:16.243648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e562f-1f68-49a5-9a5f-78c0b7282f06 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:16.380113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e562f-1f68-49a5-9a5f-78c0b7282f06 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:16.427644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959727e-a331-4f59-a0b3-f945d272d396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:16.428249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959727e-a331-4f59-a0b3-f945d272d396 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:16.428520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959727e-a331-4f59-a0b3-f945d272d396 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:16.428783 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959727e-a331-4f59-a0b3-f945d272d396 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:16.430059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959727e-a331-4f59-a0b3-f945d272d396 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:16.430356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959727e-a331-4f59-a0b3-f945d272d396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:16.431964 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959727e-a331-4f59-a0b3-f945d272d396 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:16.444657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 507/1015] 10.222.0.79 () {56 vars in 1112 bytes} [Tue Aug 9 16:33:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:16.446068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac8cd5-4112-49cc-82e9-c90c8a75a023 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:16.560737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:16.578293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac8cd5-4112-49cc-82e9-c90c8a75a023 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:16.622888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36ba5a-713f-4a6e-84db-53f1c2210026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:16.623567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36ba5a-713f-4a6e-84db-53f1c2210026 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:16.623846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36ba5a-713f-4a6e-84db-53f1c2210026 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:16.624115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36ba5a-713f-4a6e-84db-53f1c2210026 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:16.624864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36ba5a-713f-4a6e-84db-53f1c2210026 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:16.626291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36ba5a-713f-4a6e-84db-53f1c2210026 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:16.684692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa929e87-3dca-4a83-add2-401d3f2f6259 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:16.684879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa929e87-3dca-4a83-add2-401d3f2f6259 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:16.685118 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa929e87-3dca-4a83-add2-401d3f2f6259 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:16.686691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 508/1016] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:16.692335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76fd9-c14c-4c50-8ecb-fd122d23890c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:16.699159 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76fd9-c14c-4c50-8ecb-fd122d23890c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:16.699159 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76fd9-c14c-4c50-8ecb-fd122d23890c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:16.713014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0ad76fd9-c14c-4c50-8ecb-fd122d23890c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:16.714432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0ad76fd9-c14c-4c50-8ecb-fd122d23890c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:16.741495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:16.764304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 509/1017] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:33:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:33:16.794387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb4c40-1af6-4d6d-9d57-2414846a14bc None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:16.794728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb4c40-1af6-4d6d-9d57-2414846a14bc None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:16.794938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acb4c40-1af6-4d6d-9d57-2414846a14bc None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:16.796496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 510/1018] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:16.802370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874ee1c-987c-4de4-a9e6-19c698c1fe80 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:16.802724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874ee1c-987c-4de4-a9e6-19c698c1fe80 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:16.802930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874ee1c-987c-4de4-a9e6-19c698c1fe80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:16.810073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 509/1019] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:16.826346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-3874ee1c-987c-4de4-a9e6-19c698c1fe80 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:16.827662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3874ee1c-987c-4de4-a9e6-19c698c1fe80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:16.830417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76fd9-c14c-4c50-8ecb-fd122d23890c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:16.931342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 511/1020] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:16.947380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874ee1c-987c-4de4-a9e6-19c698c1fe80 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:16.962230 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76fd9-c14c-4c50-8ecb-fd122d23890c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:17.015130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6786434-1c0b-486c-a7ce-1c95c31508b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:17.016677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6786434-1c0b-486c-a7ce-1c95c31508b2 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:17.016677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6786434-1c0b-486c-a7ce-1c95c31508b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:17.016677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6786434-1c0b-486c-a7ce-1c95c31508b2 None admin] PATH_INFO: `/v3/services/image` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:17.017996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6786434-1c0b-486c-a7ce-1c95c31508b2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:17.018683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6786434-1c0b-486c-a7ce-1c95c31508b2 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-a6786434-1c0b-486c-a7ce-1c95c31508b2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:17.028322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:17.030723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:17.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 510/1021] 10.222.0.79 () {56 vars in 1105 bytes} [Tue Aug 9 16:33:16 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:17.037053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6786434-1c0b-486c-a7ce-1c95c31508b2 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:17.046380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:17.063143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:17.090216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874ee1c-987c-4de4-a9e6-19c698c1fe80 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:17.137137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea5e51-bad0-4d39-9277-0812f60dd449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:17.137854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea5e51-bad0-4d39-9277-0812f60dd449 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:17.138685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea5e51-bad0-4d39-9277-0812f60dd449 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:17.139156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea5e51-bad0-4d39-9277-0812f60dd449 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:17.140859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea5e51-bad0-4d39-9277-0812f60dd449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:17.142322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ea5e51-bad0-4d39-9277-0812f60dd449 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:17.144003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 512/1022] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:17.150092 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea5e51-bad0-4d39-9277-0812f60dd449 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:17.171943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6786434-1c0b-486c-a7ce-1c95c31508b2 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:17.225088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a214d-3b0c-47bf-974b-c44118085286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:17.225704 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a214d-3b0c-47bf-974b-c44118085286 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:17.226037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a214d-3b0c-47bf-974b-c44118085286 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:17.226302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a214d-3b0c-47bf-974b-c44118085286 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:17.226960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a214d-3b0c-47bf-974b-c44118085286 None admin] RBAC: Adding query filter params (name=image) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:17.227213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a214d-3b0c-47bf-974b-c44118085286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:17.228905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a214d-3b0c-47bf-974b-c44118085286 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:17.240412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 511/1023] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:17.242457 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a214d-3b0c-47bf-974b-c44118085286 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:17.287485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea5e51-bad0-4d39-9277-0812f60dd449 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:17.328984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7efe3e-602f-43c6-a10f-77954f7016c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:17.329983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7efe3e-602f-43c6-a10f-77954f7016c0 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:17.331028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7efe3e-602f-43c6-a10f-77954f7016c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:17.331527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7efe3e-602f-43c6-a10f-77954f7016c0 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:17.332380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7efe3e-602f-43c6-a10f-77954f7016c0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:17.334018 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7efe3e-602f-43c6-a10f-77954f7016c0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:17.345899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:17.351156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:17.373790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a214d-3b0c-47bf-974b-c44118085286 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:17.424384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496775c-3f4d-4f10-99bb-2ee4dba1bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:17.424983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496775c-3f4d-4f10-99bb-2ee4dba1bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:17.425253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496775c-3f4d-4f10-99bb-2ee4dba1bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:17.425518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496775c-3f4d-4f10-99bb-2ee4dba1bbe7 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:17.426569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496775c-3f4d-4f10-99bb-2ee4dba1bbe7 None admin] RBAC: Adding query filter params (type=image) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:17.427269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496775c-3f4d-4f10-99bb-2ee4dba1bbe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:17.429648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496775c-3f4d-4f10-99bb-2ee4dba1bbe7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:17.443441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 513/1024] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:33:17 2022] POST /identity/v3/projects => generated 311 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:33:17.443709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 512/1025] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:17.445840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7efe3e-602f-43c6-a10f-77954f7016c0 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:17.608214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7efe3e-602f-43c6-a10f-77954f7016c0 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:17.662146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf4273-3898-4b10-9685-4fbe75c62dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:17.662870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf4273-3898-4b10-9685-4fbe75c62dba None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:17.663243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf4273-3898-4b10-9685-4fbe75c62dba None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:17.663655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf4273-3898-4b10-9685-4fbe75c62dba None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:17.664455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf4273-3898-4b10-9685-4fbe75c62dba None admin] RBAC: Adding query filter params (interface=public, service_id=eedd6daefce44a61a7805e61050bd79d, region_id=RegionOne) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:17.664842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf4273-3898-4b10-9685-4fbe75c62dba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:17.666862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf4273-3898-4b10-9685-4fbe75c62dba None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:17.674930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 514/1026] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Aug 9 16:33:17 2022] GET /identity/v3/endpoints?service_id=eedd6daefce44a61a7805e61050bd79d&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:17.682178 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496775c-3f4d-4f10-99bb-2ee4dba1bbe7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:17.815564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496775c-3f4d-4f10-99bb-2ee4dba1bbe7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:17.856628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8a3a9-35eb-416b-9935-0559ac2df3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:17.857539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8a3a9-35eb-416b-9935-0559ac2df3f8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:17.858007 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8a3a9-35eb-416b-9935-0559ac2df3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:17.858471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8a3a9-35eb-416b-9935-0559ac2df3f8 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:17.859247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8a3a9-35eb-416b-9935-0559ac2df3f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:17.860784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8a3a9-35eb-416b-9935-0559ac2df3f8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:17.873989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 513/1027] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Aug 9 16:33:17 2022] GET /identity/v3/services => generated 1957 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:33:19.095374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdacff-7192-42d5-ab93-505fefa32469 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:19.095940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdacff-7192-42d5-ab93-505fefa32469 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:19.095940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdacff-7192-42d5-ab93-505fefa32469 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:19.097458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 515/1028] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:19.105010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0edcfe-a468-41e4-9d53-430e16d0579c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:19.105534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0edcfe-a468-41e4-9d53-430e16d0579c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:19.105820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0edcfe-a468-41e4-9d53-430e16d0579c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:19.127603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-3d0edcfe-a468-41e4-9d53-430e16d0579c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:19.128820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3d0edcfe-a468-41e4-9d53-430e16d0579c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:19.134264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:19.228803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 514/1029] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:19.252478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdacff-7192-42d5-ab93-505fefa32469 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:19.319340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:19.391951 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdacff-7192-42d5-ab93-505fefa32469 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:19.441048 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91893a-2204-49c1-9d01-cc0247a80a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:19.441687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91893a-2204-49c1-9d01-cc0247a80a73 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:19.441991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91893a-2204-49c1-9d01-cc0247a80a73 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:19.442256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91893a-2204-49c1-9d01-cc0247a80a73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:19.455860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91893a-2204-49c1-9d01-cc0247a80a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-ef91893a-2204-49c1-9d01-cc0247a80a73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:19.463141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:19.470964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:19.472428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 516/1030] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Aug 9 16:33:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:19.477299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0edcfe-a468-41e4-9d53-430e16d0579c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:19.627396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0edcfe-a468-41e4-9d53-430e16d0579c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:19.687599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff421ab-d1c9-4b5a-ac0e-4db672b7b539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:19.688187 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff421ab-d1c9-4b5a-ac0e-4db672b7b539 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:19.688455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff421ab-d1c9-4b5a-ac0e-4db672b7b539 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:19.688720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff421ab-d1c9-4b5a-ac0e-4db672b7b539 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:19.689392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff421ab-d1c9-4b5a-ac0e-4db672b7b539 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:19.689692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff421ab-d1c9-4b5a-ac0e-4db672b7b539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:19.691108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff421ab-d1c9-4b5a-ac0e-4db672b7b539 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:19.705709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 515/1031] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Aug 9 16:33:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:19.707599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024c65-8e5f-4eaa-9603-ad64926d3f04 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:19.707885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024c65-8e5f-4eaa-9603-ad64926d3f04 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:19.708158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024c65-8e5f-4eaa-9603-ad64926d3f04 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:19.708567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff421ab-d1c9-4b5a-ac0e-4db672b7b539 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:19.709690 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 517/1032] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:19.715550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06333dea-b2c3-420c-8c7a-577fe876692e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:19.715837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06333dea-b2c3-420c-8c7a-577fe876692e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:19.716098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06333dea-b2c3-420c-8c7a-577fe876692e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:19.720029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:19.741937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-06333dea-b2c3-420c-8c7a-577fe876692e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:19.743228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-06333dea-b2c3-420c-8c7a-577fe876692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:19.850530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 518/1033] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:19.865559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff421ab-d1c9-4b5a-ac0e-4db672b7b539 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:19.870526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06333dea-b2c3-420c-8c7a-577fe876692e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:19.919781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9200e-9863-40d7-a782-3e7a51c6ae3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:19.922072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9200e-9863-40d7-a782-3e7a51c6ae3f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:19.922072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9200e-9863-40d7-a782-3e7a51c6ae3f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:19.922072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9200e-9863-40d7-a782-3e7a51c6ae3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:19.938367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:19.943769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9200e-9863-40d7-a782-3e7a51c6ae3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:19.949562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9200e-9863-40d7-a782-3e7a51c6ae3f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-f5a9200e-9863-40d7-a782-3e7a51c6ae3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:19.964401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:19.965430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:19.966390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:19.966390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:19.966390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:19.966390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 516/1034] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:19.972478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9200e-9863-40d7-a782-3e7a51c6ae3f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:20.018492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06333dea-b2c3-420c-8c7a-577fe876692e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:20.057214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:20.074549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17add80-22c9-44fc-8607-4acf7018f1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:20.075279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17add80-22c9-44fc-8607-4acf7018f1f6 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:20.075479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17add80-22c9-44fc-8607-4acf7018f1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:20.075759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17add80-22c9-44fc-8607-4acf7018f1f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:20.077022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17add80-22c9-44fc-8607-4acf7018f1f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:20.078519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17add80-22c9-44fc-8607-4acf7018f1f6 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:20.080222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 519/1035] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:20.081453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aa755-df0b-4503-a5ef-dc249705c773 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:20.081887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aa755-df0b-4503-a5ef-dc249705c773 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:20.082042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aa755-df0b-4503-a5ef-dc249705c773 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:20.083429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 520/1036] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:20.087504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2aa755-df0b-4503-a5ef-dc249705c773 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:20.119744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9200e-9863-40d7-a782-3e7a51c6ae3f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:20.158160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f237d-0df2-4425-a1d5-d7b58085f174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:20.158757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f237d-0df2-4425-a1d5-d7b58085f174 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:20.159028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f237d-0df2-4425-a1d5-d7b58085f174 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:20.159289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f237d-0df2-4425-a1d5-d7b58085f174 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:20.159931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f237d-0df2-4425-a1d5-d7b58085f174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:20.160223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f237d-0df2-4425-a1d5-d7b58085f174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:20.161605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f237d-0df2-4425-a1d5-d7b58085f174 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:20.182981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 517/1037] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:20.184113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928793c-ae0d-4ca6-b09d-8e6364490644 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:20.184415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928793c-ae0d-4ca6-b09d-8e6364490644 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:20.184822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e928793c-ae0d-4ca6-b09d-8e6364490644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:20.207691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-e928793c-ae0d-4ca6-b09d-8e6364490644 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:20.208825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e928793c-ae0d-4ca6-b09d-8e6364490644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:20.225211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2aa755-df0b-4503-a5ef-dc249705c773 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:20.239376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:20.263342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5fabe-404d-407c-8d95-ed6323410b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:20.263947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5fabe-404d-407c-8d95-ed6323410b9f None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:20.264219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5fabe-404d-407c-8d95-ed6323410b9f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:20.264479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5fabe-404d-407c-8d95-ed6323410b9f None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:20.265124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5fabe-404d-407c-8d95-ed6323410b9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:20.270762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5fabe-404d-407c-8d95-ed6323410b9f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:20.293842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 518/1038] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:20.294828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928793c-ae0d-4ca6-b09d-8e6364490644 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:20.349232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:20.356367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 521/1039] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:33:20 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:33:20.356367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5fabe-404d-407c-8d95-ed6323410b9f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:20.444541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e928793c-ae0d-4ca6-b09d-8e6364490644 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:20.509693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae876ee-3ef4-4525-bfaf-7d8b1a3cad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:20.510945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae876ee-3ef4-4525-bfaf-7d8b1a3cad19 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:20.511451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae876ee-3ef4-4525-bfaf-7d8b1a3cad19 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:20.511944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae876ee-3ef4-4525-bfaf-7d8b1a3cad19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:20.525478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5fabe-404d-407c-8d95-ed6323410b9f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:20.526788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae876ee-3ef4-4525-bfaf-7d8b1a3cad19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:20.529996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae876ee-3ef4-4525-bfaf-7d8b1a3cad19 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-4ae876ee-3ef4-4525-bfaf-7d8b1a3cad19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:33:20.556199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:20.558409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:20.561368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 519/1040] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Aug 9 16:33:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:20.568091 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae876ee-3ef4-4525-bfaf-7d8b1a3cad19 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:20.612619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf5dcf-e289-4b44-a00e-bcd0197080fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:20.613054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf5dcf-e289-4b44-a00e-bcd0197080fb None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:20.613559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf5dcf-e289-4b44-a00e-bcd0197080fb None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:20.614734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf5dcf-e289-4b44-a00e-bcd0197080fb None admin] PATH_INFO: `/v3/services/image` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:20.614963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf5dcf-e289-4b44-a00e-bcd0197080fb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:20.617710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf5dcf-e289-4b44-a00e-bcd0197080fb None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-9abf5dcf-e289-4b44-a00e-bcd0197080fb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:20.632811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:20.637723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:20.640625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 522/1041] 10.222.0.79 () {56 vars in 1105 bytes} [Tue Aug 9 16:33:20 2022] GET /identity/v3/services/image => generated 86 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:20.648183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf5dcf-e289-4b44-a00e-bcd0197080fb None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:20.734969 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:20.808068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae876ee-3ef4-4525-bfaf-7d8b1a3cad19 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:20.814884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:20.887970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf5dcf-e289-4b44-a00e-bcd0197080fb None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:20.893032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28f114-0716-4dbb-b111-d0542964d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:20.894175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28f114-0716-4dbb-b111-d0542964d79c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:20.894658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28f114-0716-4dbb-b111-d0542964d79c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:20.895108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28f114-0716-4dbb-b111-d0542964d79c None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:20.896561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28f114-0716-4dbb-b111-d0542964d79c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:20.897053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28f114-0716-4dbb-b111-d0542964d79c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:20.899956 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28f114-0716-4dbb-b111-d0542964d79c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:20.942927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 520/1042] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Aug 9 16:33:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:20.952529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28f114-0716-4dbb-b111-d0542964d79c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:20.979315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e419979-75f6-4dc0-8abe-587edfbf304e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:20.980309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e419979-75f6-4dc0-8abe-587edfbf304e None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:20.980675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e419979-75f6-4dc0-8abe-587edfbf304e None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:20.981107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e419979-75f6-4dc0-8abe-587edfbf304e None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:20.982273 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e419979-75f6-4dc0-8abe-587edfbf304e None admin] RBAC: Adding query filter params (name=image) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:20.982896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e419979-75f6-4dc0-8abe-587edfbf304e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:20.985074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e419979-75f6-4dc0-8abe-587edfbf304e None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:20.995994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 523/1043] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:21.005206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e419979-75f6-4dc0-8abe-587edfbf304e None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:21.164150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28f114-0716-4dbb-b111-d0542964d79c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:21.203605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:21.217791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e419979-75f6-4dc0-8abe-587edfbf304e None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:21.239134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ab935-2ec7-4554-aaf4-3babad009780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:21.240156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ab935-2ec7-4554-aaf4-3babad009780 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:21.240610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ab935-2ec7-4554-aaf4-3babad009780 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:21.241374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ab935-2ec7-4554-aaf4-3babad009780 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:21.293741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163a867-b45d-49f3-ad5a-6c2fdd0ca9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:21.293741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163a867-b45d-49f3-ad5a-6c2fdd0ca9ba None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:21.293741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163a867-b45d-49f3-ad5a-6c2fdd0ca9ba None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:21.293741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163a867-b45d-49f3-ad5a-6c2fdd0ca9ba None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:21.294574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163a867-b45d-49f3-ad5a-6c2fdd0ca9ba None admin] RBAC: Adding query filter params (type=image) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:21.294574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163a867-b45d-49f3-ad5a-6c2fdd0ca9ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:21.295884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163a867-b45d-49f3-ad5a-6c2fdd0ca9ba None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:21.308640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 524/1044] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:21.309303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:21.310334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163a867-b45d-49f3-ad5a-6c2fdd0ca9ba None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:21.355091 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ab935-2ec7-4554-aaf4-3babad009780 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:21.356565 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ab935-2ec7-4554-aaf4-3babad009780 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:21.445824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163a867-b45d-49f3-ad5a-6c2fdd0ca9ba None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:21.502545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd86f47-bce9-4dc1-bc92-3f13ae7e99f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:21.503171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd86f47-bce9-4dc1-bc92-3f13ae7e99f1 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:21.503447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd86f47-bce9-4dc1-bc92-3f13ae7e99f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:21.503717 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd86f47-bce9-4dc1-bc92-3f13ae7e99f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:21.504407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd86f47-bce9-4dc1-bc92-3f13ae7e99f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:21.509591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd86f47-bce9-4dc1-bc92-3f13ae7e99f1 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:21.509740 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 521/1045] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 16:33:21.519017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ab935-2ec7-4554-aaf4-3babad009780 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:21.596206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:21.633033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 525/1046] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:33:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:33:21.656300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ab935-2ec7-4554-aaf4-3babad009780 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:21.709877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6746862-c3a9-45d4-a9dd-9629160a525a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:21.710572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6746862-c3a9-45d4-a9dd-9629160a525a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:21.710869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6746862-c3a9-45d4-a9dd-9629160a525a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:21.711166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6746862-c3a9-45d4-a9dd-9629160a525a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:21.723934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6746862-c3a9-45d4-a9dd-9629160a525a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:21.725464 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6746862-c3a9-45d4-a9dd-9629160a525a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-b6746862-c3a9-45d4-a9dd-9629160a525a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:21.749896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:21.751060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:21.752014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:21.752014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:21.752014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:21.753098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 522/1047] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:21.758079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd86f47-bce9-4dc1-bc92-3f13ae7e99f1 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:21.780592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:21.893561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd86f47-bce9-4dc1-bc92-3f13ae7e99f1 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:21.955496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cf515-b499-406f-b725-edd0d16a9eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:21.956991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cf515-b499-406f-b725-edd0d16a9eb0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:21.956991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cf515-b499-406f-b725-edd0d16a9eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:21.957431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cf515-b499-406f-b725-edd0d16a9eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:21.958604 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cf515-b499-406f-b725-edd0d16a9eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:21.959124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cf515-b499-406f-b725-edd0d16a9eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:21.961710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cf515-b499-406f-b725-edd0d16a9eb0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:21.986037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 526/1048] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:21.990112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6746862-c3a9-45d4-a9dd-9629160a525a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:22.119705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6746862-c3a9-45d4-a9dd-9629160a525a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:22.131263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:22.157002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bc1d2-9e58-436e-8f81-be63472b0712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:22.157623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bc1d2-9e58-436e-8f81-be63472b0712 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:22.157896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bc1d2-9e58-436e-8f81-be63472b0712 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:22.158263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bc1d2-9e58-436e-8f81-be63472b0712 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:22.173831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bc1d2-9e58-436e-8f81-be63472b0712 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:22.175336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bc1d2-9e58-436e-8f81-be63472b0712 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-817bc1d2-9e58-436e-8f81-be63472b0712 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:33:22.189452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:22.191321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:22.192324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:33:22.192324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:33:22.192324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:33:22.192324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:33:22.192324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:33:22.192324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:33:22.192324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:22.192324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:22.192324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 523/1049] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:33:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:22.200673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cf515-b499-406f-b725-edd0d16a9eb0 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:22.312504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:22.344585 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cf515-b499-406f-b725-edd0d16a9eb0 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:22.389415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28664e4-7640-4b0c-b655-67e1a99c8194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:22.390052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28664e4-7640-4b0c-b655-67e1a99c8194 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:22.390332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28664e4-7640-4b0c-b655-67e1a99c8194 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:22.390601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28664e4-7640-4b0c-b655-67e1a99c8194 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:22.391307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28664e4-7640-4b0c-b655-67e1a99c8194 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:22.391608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28664e4-7640-4b0c-b655-67e1a99c8194 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:22.393235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28664e4-7640-4b0c-b655-67e1a99c8194 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:22.414057 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 527/1050] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:33:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:22.419009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bc1d2-9e58-436e-8f81-be63472b0712 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:22.577264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bc1d2-9e58-436e-8f81-be63472b0712 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:22.631578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710911f-ac0b-4579-9f27-cdc651174b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:22.632277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710911f-ac0b-4579-9f27-cdc651174b10 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:22.632583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710911f-ac0b-4579-9f27-cdc651174b10 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:22.633010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710911f-ac0b-4579-9f27-cdc651174b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:22.634379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710911f-ac0b-4579-9f27-cdc651174b10 None admin] RBAC: Adding query filter params (role.id=f4cfecbeb0f44d2aa4a275e60038fa7c, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=5d17aa4b23e04290b6c8b1924b0dcf32) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:22.634729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710911f-ac0b-4579-9f27-cdc651174b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:22.636778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710911f-ac0b-4579-9f27-cdc651174b10 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:22.652478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 524/1051] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:33:22 2022] GET /identity/v3/role_assignments?user.id=5d17aa4b23e04290b6c8b1924b0dcf32&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=f4cfecbeb0f44d2aa4a275e60038fa7c => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:23.105989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ff757-8d0a-4465-95e8-810c45bfad4d None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:23.106493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ff757-8d0a-4465-95e8-810c45bfad4d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:23.106965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855ff757-8d0a-4465-95e8-810c45bfad4d None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:23.109519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 528/1052] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:23.117987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323bca-92f6-46df-994c-b08215d6e7cd None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:23.118460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323bca-92f6-46df-994c-b08215d6e7cd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:23.118915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323bca-92f6-46df-994c-b08215d6e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:23.149337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-34323bca-92f6-46df-994c-b08215d6e7cd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:23.151984 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-34323bca-92f6-46df-994c-b08215d6e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:23.199428 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:23.271920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 525/1053] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:23.312826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ff757-8d0a-4465-95e8-810c45bfad4d None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:23.492864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855ff757-8d0a-4465-95e8-810c45bfad4d None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:23.555409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df4713-cc5b-4a5c-a767-df898c679eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:23.556804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df4713-cc5b-4a5c-a767-df898c679eba None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:23.557499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df4713-cc5b-4a5c-a767-df898c679eba None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:23.558227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df4713-cc5b-4a5c-a767-df898c679eba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:23.563502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:23.574003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df4713-cc5b-4a5c-a767-df898c679eba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-72df4713-cc5b-4a5c-a767-df898c679eba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:23.579313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:23.582693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:23.583597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 529/1054] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:33:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:23.586702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34323bca-92f6-46df-994c-b08215d6e7cd None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:23.709532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34323bca-92f6-46df-994c-b08215d6e7cd None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:23.748765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a5aa8-7111-47a8-8fd0-d34690a49470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:23.749278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a5aa8-7111-47a8-8fd0-d34690a49470 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:23.749548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a5aa8-7111-47a8-8fd0-d34690a49470 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:23.749808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a5aa8-7111-47a8-8fd0-d34690a49470 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:23.750494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a5aa8-7111-47a8-8fd0-d34690a49470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:23.750787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a5aa8-7111-47a8-8fd0-d34690a49470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:23.752270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a5aa8-7111-47a8-8fd0-d34690a49470 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:23.768192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 526/1055] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:33:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:23.769976 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df4713-cc5b-4a5c-a767-df898c679eba None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:23.900725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df4713-cc5b-4a5c-a767-df898c679eba None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:23.937017 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efeb820-5466-45d4-a8c2-15e8bfbdf5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:23.937644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeb820-5466-45d4-a8c2-15e8bfbdf5d7 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:23.937917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeb820-5466-45d4-a8c2-15e8bfbdf5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:23.938216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efeb820-5466-45d4-a8c2-15e8bfbdf5d7 None admin] PATH_INFO: `/v3/users/a04b52b12cc04a9f9695810e1d95ed16` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:23.954547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efeb820-5466-45d4-a8c2-15e8bfbdf5d7 None admin] RBAC: Authorizing `identity:get_user(user_id=a04b52b12cc04a9f9695810e1d95ed16)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:23.956065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efeb820-5466-45d4-a8c2-15e8bfbdf5d7 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:23.958099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 530/1056] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:23 2022] GET /identity/v3/users/a04b52b12cc04a9f9695810e1d95ed16 => generated 285 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:23.964911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a5aa8-7111-47a8-8fd0-d34690a49470 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:23.975627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5692ece-e5e1-45c9-8e82-03de4541d1da None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:23.975967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5692ece-e5e1-45c9-8e82-03de4541d1da None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:23.976784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5692ece-e5e1-45c9-8e82-03de4541d1da None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:23.978612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 531/1057] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:23.983553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352032-de07-4436-ab51-1be637247e8b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:23.983847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352032-de07-4436-ab51-1be637247e8b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:23.984100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80352032-de07-4436-ab51-1be637247e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:24.002838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-80352032-de07-4436-ab51-1be637247e8b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:24.004014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-80352032-de07-4436-ab51-1be637247e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:24.010242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:24.082598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:24.109318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 532/1058] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:24.112012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a5aa8-7111-47a8-8fd0-d34690a49470 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:24.136233 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352032-de07-4436-ab51-1be637247e8b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:24.155715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be4015-fa69-4c4b-82ac-4b514e213e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:24.156587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4015-fa69-4c4b-82ac-4b514e213e2f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:24.157025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4015-fa69-4c4b-82ac-4b514e213e2f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:24.157421 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4015-fa69-4c4b-82ac-4b514e213e2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:24.251821 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be4015-fa69-4c4b-82ac-4b514e213e2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:24.253593 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be4015-fa69-4c4b-82ac-4b514e213e2f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:24.262354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80352032-de07-4436-ab51-1be637247e8b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:24.299902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:24.308543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd04ac-836a-49d2-a824-6ed023e930c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:24.308756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd04ac-836a-49d2-a824-6ed023e930c0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:24.308959 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd04ac-836a-49d2-a824-6ed023e930c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:24.309224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd04ac-836a-49d2-a824-6ed023e930c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:24.381875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:24.402960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd04ac-836a-49d2-a824-6ed023e930c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:24.403634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 527/1059] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:33:24.405378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd04ac-836a-49d2-a824-6ed023e930c0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:24.410064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be4015-fa69-4c4b-82ac-4b514e213e2f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:24.552336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be4015-fa69-4c4b-82ac-4b514e213e2f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:24.558024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 533/1060] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 16:33:24.561807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd04ac-836a-49d2-a824-6ed023e930c0 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:24.594925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:24.604791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66496d-fe7d-4210-b49c-eb8213e84439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:24.605567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66496d-fe7d-4210-b49c-eb8213e84439 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:24.611348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66496d-fe7d-4210-b49c-eb8213e84439 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:24.611348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66496d-fe7d-4210-b49c-eb8213e84439 None admin] PATH_INFO: `/v3/projects/73b27f49aa19494f85cd53089a1e5224` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:24.611348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66496d-fe7d-4210-b49c-eb8213e84439 None admin] RBAC: Authorizing `identity:get_project(project_id=73b27f49aa19494f85cd53089a1e5224)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:24.611917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66496d-fe7d-4210-b49c-eb8213e84439 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:24.622612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 528/1061] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:33:24 2022] GET /identity/v3/projects/73b27f49aa19494f85cd53089a1e5224 => generated 311 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:24.625207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66496d-fe7d-4210-b49c-eb8213e84439 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:24.674703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:24.692377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd04ac-836a-49d2-a824-6ed023e930c0 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:24.744557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33f7e4-8216-494e-b342-34edf39afde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:24.745257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33f7e4-8216-494e-b342-34edf39afde9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:24.745555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33f7e4-8216-494e-b342-34edf39afde9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:24.745859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33f7e4-8216-494e-b342-34edf39afde9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:24.754255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66496d-fe7d-4210-b49c-eb8213e84439 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:24.759225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33f7e4-8216-494e-b342-34edf39afde9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:24.760975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33f7e4-8216-494e-b342-34edf39afde9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-4b33f7e4-8216-494e-b342-34edf39afde9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:24.780901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:24.781877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:24.782822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:24.782822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:24.782822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:24.782822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 534/1062] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:24.790696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33f7e4-8216-494e-b342-34edf39afde9 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:24.807328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacd52ad-1aa1-4e3e-83cc-565ca1ef988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:24.808439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd52ad-1aa1-4e3e-83cc-565ca1ef988a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:24.809350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd52ad-1aa1-4e3e-83cc-565ca1ef988a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:24.809818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacd52ad-1aa1-4e3e-83cc-565ca1ef988a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:24.811625 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd52ad-1aa1-4e3e-83cc-565ca1ef988a None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=73b27f49aa19494f85cd53089a1e5224, user.id=a04b52b12cc04a9f9695810e1d95ed16) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:24.812321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd52ad-1aa1-4e3e-83cc-565ca1ef988a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:24.814266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacd52ad-1aa1-4e3e-83cc-565ca1ef988a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:24.823350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 529/1063] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:33:24 2022] GET /identity/v3/role_assignments?user.id=a04b52b12cc04a9f9695810e1d95ed16&scope.project.id=73b27f49aa19494f85cd53089a1e5224&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:24.948580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33f7e4-8216-494e-b342-34edf39afde9 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:24.994834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be56b9-211b-4bb4-ab11-3f92d58411cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:24.995463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be56b9-211b-4bb4-ab11-3f92d58411cd None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:24.995751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be56b9-211b-4bb4-ab11-3f92d58411cd None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:24.996072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be56b9-211b-4bb4-ab11-3f92d58411cd None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:24.996783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be56b9-211b-4bb4-ab11-3f92d58411cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:24.997537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be56b9-211b-4bb4-ab11-3f92d58411cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:24.998878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be56b9-211b-4bb4-ab11-3f92d58411cd None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:25.002083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:25.015073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 535/1064] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:25.055263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e669ab0-6d5c-4270-86d4-67421f1dd9d1 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:25.055740 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e669ab0-6d5c-4270-86d4-67421f1dd9d1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:25.056151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e669ab0-6d5c-4270-86d4-67421f1dd9d1 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:25.057937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 530/1065] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:25.064490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9312d7e-9aa8-492e-9063-84ac76b4b690 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:25.064921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9312d7e-9aa8-492e-9063-84ac76b4b690 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:25.065032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9312d7e-9aa8-492e-9063-84ac76b4b690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:25.090381 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-b9312d7e-9aa8-492e-9063-84ac76b4b690 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:25.092024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b9312d7e-9aa8-492e-9063-84ac76b4b690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:25.201275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 536/1066] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:25.231461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e669ab0-6d5c-4270-86d4-67421f1dd9d1 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:25.312598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:25.370448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e669ab0-6d5c-4270-86d4-67421f1dd9d1 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:25.408918 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d53d8b-f90f-403a-a494-df35d9943bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:25.409579 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53d8b-f90f-403a-a494-df35d9943bed None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:25.409853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53d8b-f90f-403a-a494-df35d9943bed None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:25.410158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d53d8b-f90f-403a-a494-df35d9943bed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:25.431762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d53d8b-f90f-403a-a494-df35d9943bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-90d53d8b-f90f-403a-a494-df35d9943bed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:25.438152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:25.439295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:25.440844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 531/1067] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Aug 9 16:33:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:25.447378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9312d7e-9aa8-492e-9063-84ac76b4b690 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:25.578901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9312d7e-9aa8-492e-9063-84ac76b4b690 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:25.612325 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:25.621942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484177b-0b9a-487e-b4f7-fd7351f12462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:25.622825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484177b-0b9a-487e-b4f7-fd7351f12462 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:25.623279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484177b-0b9a-487e-b4f7-fd7351f12462 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:25.623630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484177b-0b9a-487e-b4f7-fd7351f12462 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:25.624394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484177b-0b9a-487e-b4f7-fd7351f12462 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:25.624754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484177b-0b9a-487e-b4f7-fd7351f12462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:25.626406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484177b-0b9a-487e-b4f7-fd7351f12462 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:25.636090 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 537/1068] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Aug 9 16:33:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:25.642050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d53d8b-f90f-403a-a494-df35d9943bed None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:25.770705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d53d8b-f90f-403a-a494-df35d9943bed None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:25.793095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:25.817873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f97e4f-c849-4149-976f-465b9b1057a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:25.818952 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f97e4f-c849-4149-976f-465b9b1057a2 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:25.819404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f97e4f-c849-4149-976f-465b9b1057a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:25.820368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f97e4f-c849-4149-976f-465b9b1057a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:25.836559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f97e4f-c849-4149-976f-465b9b1057a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:25.837894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f97e4f-c849-4149-976f-465b9b1057a2 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-e5f97e4f-c849-4149-976f-465b9b1057a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:25.856541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:25.858426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:25.860732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:25.860732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:25.860732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:25.860732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 532/1069] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:25.874364 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484177b-0b9a-487e-b4f7-fd7351f12462 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:26.002312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484177b-0b9a-487e-b4f7-fd7351f12462 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:26.043180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10212df-8eb9-4e57-acd5-e78ba6eef7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:26.043802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10212df-8eb9-4e57-acd5-e78ba6eef7e9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:26.044079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10212df-8eb9-4e57-acd5-e78ba6eef7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:26.044346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10212df-8eb9-4e57-acd5-e78ba6eef7e9 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:26.045020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10212df-8eb9-4e57-acd5-e78ba6eef7e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:26.045316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10212df-8eb9-4e57-acd5-e78ba6eef7e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:26.046805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10212df-8eb9-4e57-acd5-e78ba6eef7e9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:26.066410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 538/1070] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:26.067606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f97e4f-c849-4149-976f-465b9b1057a2 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:26.174199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f97e4f-c849-4149-976f-465b9b1057a2 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:26.210118 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f8639-8bc5-4066-a837-66cc1b0c1d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:26.211263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8639-8bc5-4066-a837-66cc1b0c1d69 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:26.211538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8639-8bc5-4066-a837-66cc1b0c1d69 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:26.211822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f8639-8bc5-4066-a837-66cc1b0c1d69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:26.220531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f8639-8bc5-4066-a837-66cc1b0c1d69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:26.222260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f8639-8bc5-4066-a837-66cc1b0c1d69 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-f54f8639-8bc5-4066-a837-66cc1b0c1d69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:33:26.236536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:26.237520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:26.238658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 533/1071] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Aug 9 16:33:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:26.243904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10212df-8eb9-4e57-acd5-e78ba6eef7e9 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:26.296206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:26.346140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10212df-8eb9-4e57-acd5-e78ba6eef7e9 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:26.383259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932c48b-d532-4cc6-8255-dc2660a105be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:26.383851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932c48b-d532-4cc6-8255-dc2660a105be None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:26.384122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932c48b-d532-4cc6-8255-dc2660a105be None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:26.384383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932c48b-d532-4cc6-8255-dc2660a105be None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:26.385104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932c48b-d532-4cc6-8255-dc2660a105be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:26.385397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932c48b-d532-4cc6-8255-dc2660a105be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:26.386845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932c48b-d532-4cc6-8255-dc2660a105be None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:26.405510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 539/1072] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Aug 9 16:33:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:26.411657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f8639-8bc5-4066-a837-66cc1b0c1d69 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:26.444815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:26.518752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f8639-8bc5-4066-a837-66cc1b0c1d69 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:26.559525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583f700-e65f-4caa-8b63-a36638d1240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:26.560146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583f700-e65f-4caa-8b63-a36638d1240b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:26.560414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583f700-e65f-4caa-8b63-a36638d1240b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:26.560842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583f700-e65f-4caa-8b63-a36638d1240b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:26.573159 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583f700-e65f-4caa-8b63-a36638d1240b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:26.574915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583f700-e65f-4caa-8b63-a36638d1240b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-0583f700-e65f-4caa-8b63-a36638d1240b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:26.590907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:26.592278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:26.593168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:26.593168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:26.593168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:26.593168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 534/1073] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:26.598625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932c48b-d532-4cc6-8255-dc2660a105be None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:26.724456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932c48b-d532-4cc6-8255-dc2660a105be None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:26.838226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:26.867349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f08ce9-184e-49d5-be49-19ce65a2bd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:26.868200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08ce9-184e-49d5-be49-19ce65a2bd4b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:26.868603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08ce9-184e-49d5-be49-19ce65a2bd4b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:26.868949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f08ce9-184e-49d5-be49-19ce65a2bd4b None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:26.869592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f08ce9-184e-49d5-be49-19ce65a2bd4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:26.869903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f08ce9-184e-49d5-be49-19ce65a2bd4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:26.871291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f08ce9-184e-49d5-be49-19ce65a2bd4b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:26.886760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 540/1074] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:26.892016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583f700-e65f-4caa-8b63-a36638d1240b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:26.983477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:27.001369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0a3e5-9ef8-43b0-a4eb-a4479c176ee6 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:27.012201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0a3e5-9ef8-43b0-a4eb-a4479c176ee6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:27.012201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0a3e5-9ef8-43b0-a4eb-a4479c176ee6 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:27.012201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 541/1075] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:27.012696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583f700-e65f-4caa-8b63-a36638d1240b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:27.012856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e04ca-c47a-4d96-84e7-d39ecc5baf94 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:27.012856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e04ca-c47a-4d96-84e7-d39ecc5baf94 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:27.012856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e04ca-c47a-4d96-84e7-d39ecc5baf94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:27.029856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-513e04ca-c47a-4d96-84e7-d39ecc5baf94 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:27.031191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-513e04ca-c47a-4d96-84e7-d39ecc5baf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:27.041614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1338047-2364-4ac2-9f50-9c54b92d48c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:27.042276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1338047-2364-4ac2-9f50-9c54b92d48c7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:27.042608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1338047-2364-4ac2-9f50-9c54b92d48c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:27.042884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1338047-2364-4ac2-9f50-9c54b92d48c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:27.050060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1338047-2364-4ac2-9f50-9c54b92d48c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:27.051504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1338047-2364-4ac2-9f50-9c54b92d48c7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-e1338047-2364-4ac2-9f50-9c54b92d48c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:33:27.068619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:27.070369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:27.071309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:33:27.071309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:33:27.071309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:33:27.071309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:33:27.071309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:33:27.071309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:33:27.071309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:27.071309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:27.071309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 535/1076] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:33:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:27.076307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1338047-2364-4ac2-9f50-9c54b92d48c7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:27.111511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 542/1077] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:27.135801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e04ca-c47a-4d96-84e7-d39ecc5baf94 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:27.196567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1338047-2364-4ac2-9f50-9c54b92d48c7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:27.234316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:27.234539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5b643-7899-4dfb-a8f1-441945903b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:27.234868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5b643-7899-4dfb-a8f1-441945903b9f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:27.235160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5b643-7899-4dfb-a8f1-441945903b9f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:27.235444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5b643-7899-4dfb-a8f1-441945903b9f None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:27.236160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5b643-7899-4dfb-a8f1-441945903b9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:27.236477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5b643-7899-4dfb-a8f1-441945903b9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:27.238162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5b643-7899-4dfb-a8f1-441945903b9f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:27.240459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e04ca-c47a-4d96-84e7-d39ecc5baf94 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:27.252570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 536/1078] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:27.259228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5b643-7899-4dfb-a8f1-441945903b9f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:27.262183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:27.279208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9cc1b-71d8-4388-b6fc-a8fc5ba57619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:27.279843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9cc1b-71d8-4388-b6fc-a8fc5ba57619 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:27.280249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9cc1b-71d8-4388-b6fc-a8fc5ba57619 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:27.280707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9cc1b-71d8-4388-b6fc-a8fc5ba57619 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:27.293107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9cc1b-71d8-4388-b6fc-a8fc5ba57619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-07e9cc1b-71d8-4388-b6fc-a8fc5ba57619 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:27.299119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:27.300114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:27.307190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 543/1079] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:33:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:27.307878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9cc1b-71d8-4388-b6fc-a8fc5ba57619 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:27.365697 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5b643-7899-4dfb-a8f1-441945903b9f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:27.401877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23305e75-7a2f-4810-a0ce-320fee6f36b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:27.402562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305e75-7a2f-4810-a0ce-320fee6f36b4 None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:27.402833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305e75-7a2f-4810-a0ce-320fee6f36b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:27.403096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23305e75-7a2f-4810-a0ce-320fee6f36b4 None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/5d17aa4b23e04290b6c8b1924b0dcf32/roles/f4cfecbeb0f44d2aa4a275e60038fa7c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:27.405811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23305e75-7a2f-4810-a0ce-320fee6f36b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=5d17aa4b23e04290b6c8b1924b0dcf32, role_id=f4cfecbeb0f44d2aa4a275e60038fa7c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:27.407622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23305e75-7a2f-4810-a0ce-320fee6f36b4 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:27.417188 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9cc1b-71d8-4388-b6fc-a8fc5ba57619 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:27.454632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbe10d-418c-4126-b866-5fcf8cf789e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:27.455250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbe10d-418c-4126-b866-5fcf8cf789e9 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:27.455581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbe10d-418c-4126-b866-5fcf8cf789e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:27.455888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbe10d-418c-4126-b866-5fcf8cf789e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:27.456708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbe10d-418c-4126-b866-5fcf8cf789e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:27.457085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbe10d-418c-4126-b866-5fcf8cf789e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:27.458684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbe10d-418c-4126-b866-5fcf8cf789e9 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:27.468863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 544/1080] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:33:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:27.474985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbe10d-418c-4126-b866-5fcf8cf789e9 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:27.497452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:27.525353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 537/1081] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:33:27 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/5d17aa4b23e04290b6c8b1924b0dcf32/roles/f4cfecbeb0f44d2aa4a275e60038fa7c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:33:27.592613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbe10d-418c-4126-b866-5fcf8cf789e9 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:27.635662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33198e08-2164-4b95-8994-fc6f9f9bb817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:27.636680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33198e08-2164-4b95-8994-fc6f9f9bb817 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:27.636680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33198e08-2164-4b95-8994-fc6f9f9bb817 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:27.636869 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33198e08-2164-4b95-8994-fc6f9f9bb817 None admin] PATH_INFO: `/v3/users/a04b52b12cc04a9f9695810e1d95ed16` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:27.638063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33198e08-2164-4b95-8994-fc6f9f9bb817 None admin] RBAC: Authorizing `identity:get_user(user_id=a04b52b12cc04a9f9695810e1d95ed16)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:27.639534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33198e08-2164-4b95-8994-fc6f9f9bb817 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:27.641183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 545/1082] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:27 2022] GET /identity/v3/users/a04b52b12cc04a9f9695810e1d95ed16 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:27.647692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23305e75-7a2f-4810-a0ce-320fee6f36b4 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:27.728361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:27.769126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23305e75-7a2f-4810-a0ce-320fee6f36b4 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:27.820337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03639f33-b74f-47c1-9580-3f162bd623fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:27.820973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03639f33-b74f-47c1-9580-3f162bd623fa None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:27.821251 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03639f33-b74f-47c1-9580-3f162bd623fa None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:27.821521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03639f33-b74f-47c1-9580-3f162bd623fa None admin] PATH_INFO: `/v3/projects/73b27f49aa19494f85cd53089a1e5224` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:27.822684 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03639f33-b74f-47c1-9580-3f162bd623fa None admin] RBAC: Authorizing `identity:get_project(project_id=73b27f49aa19494f85cd53089a1e5224)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:27.824757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03639f33-b74f-47c1-9580-3f162bd623fa None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:27.826496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 538/1083] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:33:27 2022] GET /identity/v3/projects/73b27f49aa19494f85cd53089a1e5224 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:27.833093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33198e08-2164-4b95-8994-fc6f9f9bb817 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:27.934653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:27.950363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33198e08-2164-4b95-8994-fc6f9f9bb817 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:27.992456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bbead-aa8f-4d66-9f2d-534b37094791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:27.993126 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bbead-aa8f-4d66-9f2d-534b37094791 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:27.993393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bbead-aa8f-4d66-9f2d-534b37094791 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:27.993659 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bbead-aa8f-4d66-9f2d-534b37094791 None admin] PATH_INFO: `/v3/projects/73b27f49aa19494f85cd53089a1e5224/users/a04b52b12cc04a9f9695810e1d95ed16/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:27.995648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90bbead-aa8f-4d66-9f2d-534b37094791 None admin] RBAC: Authorizing `identity:create_grant(project_id=73b27f49aa19494f85cd53089a1e5224, user_id=a04b52b12cc04a9f9695810e1d95ed16, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:27.997081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90bbead-aa8f-4d66-9f2d-534b37094791 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:28.077426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 546/1084] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:33:27 2022] PUT /identity/v3/projects/73b27f49aa19494f85cd53089a1e5224/users/a04b52b12cc04a9f9695810e1d95ed16/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:33:29.665201 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc75716-69ab-42bc-a3ce-f864008f38ac None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:29.665842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc75716-69ab-42bc-a3ce-f864008f38ac None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:29.665842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc75716-69ab-42bc-a3ce-f864008f38ac None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:29.667103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 539/1085] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:29.673092 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b32e7-04cc-462b-9f0d-8d08650aec6c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:29.673598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b32e7-04cc-462b-9f0d-8d08650aec6c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:29.673760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b32e7-04cc-462b-9f0d-8d08650aec6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:29.692964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-700b32e7-04cc-462b-9f0d-8d08650aec6c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:29.694183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-700b32e7-04cc-462b-9f0d-8d08650aec6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:29.821092 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 547/1086] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:29.844829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc75716-69ab-42bc-a3ce-f864008f38ac None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:29.952795 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc75716-69ab-42bc-a3ce-f864008f38ac None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:29.984860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:29.990587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94256077-b9f0-4139-a872-8dee1e24e557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:29.991219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94256077-b9f0-4139-a872-8dee1e24e557 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:29.991520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94256077-b9f0-4139-a872-8dee1e24e557 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:29.991784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94256077-b9f0-4139-a872-8dee1e24e557 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.002785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94256077-b9f0-4139-a872-8dee1e24e557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-94256077-b9f0-4139-a872-8dee1e24e557 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:30.007470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:30.008436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:30.009529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 540/1087] 10.222.0.79 () {56 vars in 1120 bytes} [Tue Aug 9 16:33:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:30.014937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b32e7-04cc-462b-9f0d-8d08650aec6c None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:30.096127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:30.120043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcc681-5c3b-4971-a146-3284366dc936 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.120576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b32e7-04cc-462b-9f0d-8d08650aec6c None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:30.120903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcc681-5c3b-4971-a146-3284366dc936 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.121183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcc681-5c3b-4971-a146-3284366dc936 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.122542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 541/1088] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:33:30.128217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e54e12-292e-4f39-bdfa-e21a3a08f27d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.128493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e54e12-292e-4f39-bdfa-e21a3a08f27d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.128763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e54e12-292e-4f39-bdfa-e21a3a08f27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.147383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-99e54e12-292e-4f39-bdfa-e21a3a08f27d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:30.149067 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-99e54e12-292e-4f39-bdfa-e21a3a08f27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:30.157006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba2e65-1d88-4bc1-8b53-0984c33dfef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:30.157608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba2e65-1d88-4bc1-8b53-0984c33dfef4 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.157881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba2e65-1d88-4bc1-8b53-0984c33dfef4 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.158174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba2e65-1d88-4bc1-8b53-0984c33dfef4 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.159263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba2e65-1d88-4bc1-8b53-0984c33dfef4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:30.159263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba2e65-1d88-4bc1-8b53-0984c33dfef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:30.160563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba2e65-1d88-4bc1-8b53-0984c33dfef4 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:30.175211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 548/1089] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Aug 9 16:33:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:30.176294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba2e65-1d88-4bc1-8b53-0984c33dfef4 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:30.220669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 542/1090] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:30.244808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e54e12-292e-4f39-bdfa-e21a3a08f27d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:30.287907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba2e65-1d88-4bc1-8b53-0984c33dfef4 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:30.325849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911caf69-bd5b-47d1-a9a1-58e8a8436329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:30.326506 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911caf69-bd5b-47d1-a9a1-58e8a8436329 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.326781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911caf69-bd5b-47d1-a9a1-58e8a8436329 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.327047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911caf69-bd5b-47d1-a9a1-58e8a8436329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.334637 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:30.340104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911caf69-bd5b-47d1-a9a1-58e8a8436329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:30.341556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911caf69-bd5b-47d1-a9a1-58e8a8436329 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:30.354002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e54e12-292e-4f39-bdfa-e21a3a08f27d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-911caf69-bd5b-47d1-a9a1-58e8a8436329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:30.356420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:30.357490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:30.358431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:30.358431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:30.358431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:30.358431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 549/1091] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:30.363567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911caf69-bd5b-47d1-a9a1-58e8a8436329 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:30.376321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:30.392135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0cd15-f72a-49cf-9d6f-3826073c6b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:30.392135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0cd15-f72a-49cf-9d6f-3826073c6b6f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.392135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0cd15-f72a-49cf-9d6f-3826073c6b6f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.392135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0cd15-f72a-49cf-9d6f-3826073c6b6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.403731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0cd15-f72a-49cf-9d6f-3826073c6b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-91a0cd15-f72a-49cf-9d6f-3826073c6b6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:30.408480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:30.409493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:30.410384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 543/1092] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:33:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:30.415877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0cd15-f72a-49cf-9d6f-3826073c6b6f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:30.482044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911caf69-bd5b-47d1-a9a1-58e8a8436329 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:30.522531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fd2a5-0782-45ae-886e-66508ca36b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:30.523878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fd2a5-0782-45ae-886e-66508ca36b87 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.524166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fd2a5-0782-45ae-886e-66508ca36b87 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.524437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fd2a5-0782-45ae-886e-66508ca36b87 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.525087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fd2a5-0782-45ae-886e-66508ca36b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:30.525387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fd2a5-0782-45ae-886e-66508ca36b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:30.526785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34fd2a5-0782-45ae-886e-66508ca36b87 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:30.534329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0cd15-f72a-49cf-9d6f-3826073c6b6f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:30.540395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 550/1093] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:30.546934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fd2a5-0782-45ae-886e-66508ca36b87 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:30.569367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ec0b1-b01f-4b37-a16a-cc86b83bf239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:30.570495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ec0b1-b01f-4b37-a16a-cc86b83bf239 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.570766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ec0b1-b01f-4b37-a16a-cc86b83bf239 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.571032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ec0b1-b01f-4b37-a16a-cc86b83bf239 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.571746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ec0b1-b01f-4b37-a16a-cc86b83bf239 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:30.572037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ec0b1-b01f-4b37-a16a-cc86b83bf239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:30.573413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ec0b1-b01f-4b37-a16a-cc86b83bf239 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:30.582447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 544/1094] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:33:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:30.588944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ec0b1-b01f-4b37-a16a-cc86b83bf239 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:30.601268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:30.633874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:30.655356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fd2a5-0782-45ae-886e-66508ca36b87 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:30.693592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459aad66-3794-42a1-8de8-f187a1abecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:30.694342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459aad66-3794-42a1-8de8-f187a1abecc2 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.695255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459aad66-3794-42a1-8de8-f187a1abecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.695525 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459aad66-3794-42a1-8de8-f187a1abecc2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.695618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ec0b1-b01f-4b37-a16a-cc86b83bf239 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:30.703124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459aad66-3794-42a1-8de8-f187a1abecc2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:30.704568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459aad66-3794-42a1-8de8-f187a1abecc2 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-459aad66-3794-42a1-8de8-f187a1abecc2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:33:30.716899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:30.718119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:30.719149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 551/1095] 10.222.0.79 () {56 vars in 1093 bytes} [Tue Aug 9 16:33:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:30.724573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459aad66-3794-42a1-8de8-f187a1abecc2 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:30.733399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad6e78-78df-47c1-87a4-ce8b9fc96599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:30.734087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad6e78-78df-47c1-87a4-ce8b9fc96599 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.734367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad6e78-78df-47c1-87a4-ce8b9fc96599 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.734636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad6e78-78df-47c1-87a4-ce8b9fc96599 None admin] PATH_INFO: `/v3/users/a04b52b12cc04a9f9695810e1d95ed16` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.735778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad6e78-78df-47c1-87a4-ce8b9fc96599 None admin] RBAC: Authorizing `identity:get_user(user_id=a04b52b12cc04a9f9695810e1d95ed16)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:30.737196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad6e78-78df-47c1-87a4-ce8b9fc96599 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:30.739120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 545/1096] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:30 2022] GET /identity/v3/users/a04b52b12cc04a9f9695810e1d95ed16 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:30.745609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad6e78-78df-47c1-87a4-ce8b9fc96599 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:30.837235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459aad66-3794-42a1-8de8-f187a1abecc2 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:30.848517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:30.851759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad6e78-78df-47c1-87a4-ce8b9fc96599 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:30.870023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:30.875112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef460e74-3456-4d41-bb1e-1ebda26180dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:30.875768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef460e74-3456-4d41-bb1e-1ebda26180dc None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.876046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef460e74-3456-4d41-bb1e-1ebda26180dc None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.876315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef460e74-3456-4d41-bb1e-1ebda26180dc None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.877044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef460e74-3456-4d41-bb1e-1ebda26180dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:30.877370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef460e74-3456-4d41-bb1e-1ebda26180dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:30.878828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef460e74-3456-4d41-bb1e-1ebda26180dc None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:30.890650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e703e4-f966-46ba-b03e-89573b7d13b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:30.891181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e703e4-f966-46ba-b03e-89573b7d13b8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:30.891453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e703e4-f966-46ba-b03e-89573b7d13b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:30.891709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e703e4-f966-46ba-b03e-89573b7d13b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:30.898507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 552/1097] 10.222.0.79 () {56 vars in 1133 bytes} [Tue Aug 9 16:33:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:30.905578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef460e74-3456-4d41-bb1e-1ebda26180dc None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:30.975632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e703e4-f966-46ba-b03e-89573b7d13b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:30.977100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e703e4-f966-46ba-b03e-89573b7d13b8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:31.018991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef460e74-3456-4d41-bb1e-1ebda26180dc None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:31.062323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8e3c2-9cb4-4594-8a87-bf097a55b86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:31.062932 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8e3c2-9cb4-4594-8a87-bf097a55b86d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:31.063222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8e3c2-9cb4-4594-8a87-bf097a55b86d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:31.063477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b8e3c2-9cb4-4594-8a87-bf097a55b86d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:31.099244 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 546/1098] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 09 16:33:31.103264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:31.106148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e703e4-f966-46ba-b03e-89573b7d13b8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:31.119546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:31.142995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8e3c2-9cb4-4594-8a87-bf097a55b86d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:31.144352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b8e3c2-9cb4-4594-8a87-bf097a55b86d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:31.230802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e703e4-f966-46ba-b03e-89573b7d13b8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:31.259821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 553/1099] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:33:31.265868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8e3c2-9cb4-4594-8a87-bf097a55b86d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:31.268661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fda4d-bb06-4b3e-8c08-a01eb81bd33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:31.269323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fda4d-bb06-4b3e-8c08-a01eb81bd33e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:31.269572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fda4d-bb06-4b3e-8c08-a01eb81bd33e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:31.269840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fda4d-bb06-4b3e-8c08-a01eb81bd33e None admin] PATH_INFO: `/v3/projects/73b27f49aa19494f85cd53089a1e5224` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:31.271011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fda4d-bb06-4b3e-8c08-a01eb81bd33e None admin] RBAC: Authorizing `identity:get_project(project_id=73b27f49aa19494f85cd53089a1e5224)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:31.272465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0fda4d-bb06-4b3e-8c08-a01eb81bd33e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:31.274195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 547/1100] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:33:31 2022] GET /identity/v3/projects/73b27f49aa19494f85cd53089a1e5224 => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:31.280507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fda4d-bb06-4b3e-8c08-a01eb81bd33e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:31.340058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:31.370185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:31.379716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b8e3c2-9cb4-4594-8a87-bf097a55b86d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:31.394121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fda4d-bb06-4b3e-8c08-a01eb81bd33e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:31.422267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e06c65-c3d0-40c5-b2ca-cef807ffde21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:31.423240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e06c65-c3d0-40c5-b2ca-cef807ffde21 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:31.423674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e06c65-c3d0-40c5-b2ca-cef807ffde21 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:31.423980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e06c65-c3d0-40c5-b2ca-cef807ffde21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:31.432368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4e10a-1f2f-4ce1-83bf-3ebb8643d6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:31.433005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4e10a-1f2f-4ce1-83bf-3ebb8643d6ee None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:31.433278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4e10a-1f2f-4ce1-83bf-3ebb8643d6ee None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:31.433545 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4e10a-1f2f-4ce1-83bf-3ebb8643d6ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:31.434301 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4e10a-1f2f-4ce1-83bf-3ebb8643d6ee None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=73b27f49aa19494f85cd53089a1e5224, user.id=a04b52b12cc04a9f9695810e1d95ed16) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:31.434605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4e10a-1f2f-4ce1-83bf-3ebb8643d6ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:31.435773 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e06c65-c3d0-40c5-b2ca-cef807ffde21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:31.436037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4e10a-1f2f-4ce1-83bf-3ebb8643d6ee None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:31.437212 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e06c65-c3d0-40c5-b2ca-cef807ffde21 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:31.442151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 548/1101] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:33:31 2022] GET /identity/v3/role_assignments?user.id=a04b52b12cc04a9f9695810e1d95ed16&scope.project.id=73b27f49aa19494f85cd53089a1e5224&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-07e06c65-c3d0-40c5-b2ca-cef807ffde21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:33:31.455683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:31.457254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:33:31.458557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:33:31.458557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:33:31.458557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:31.458557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 554/1102] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:33:31.464108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4e10a-1f2f-4ce1-83bf-3ebb8643d6ee None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:31.593399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4e10a-1f2f-4ce1-83bf-3ebb8643d6ee None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:31.643726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab24a1-50ba-4cbb-9714-4a2627a670ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:31.644675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab24a1-50ba-4cbb-9714-4a2627a670ef None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:31.645058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab24a1-50ba-4cbb-9714-4a2627a670ef None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:31.645489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab24a1-50ba-4cbb-9714-4a2627a670ef None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:31.646571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab24a1-50ba-4cbb-9714-4a2627a670ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:31.647037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab24a1-50ba-4cbb-9714-4a2627a670ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:31.649852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab24a1-50ba-4cbb-9714-4a2627a670ef None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:31.670863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 549/1103] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:31.676322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e06c65-c3d0-40c5-b2ca-cef807ffde21 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:31.789203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e06c65-c3d0-40c5-b2ca-cef807ffde21 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:31.813156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:31.828531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421daa0e-2380-455a-90d9-e4b213e6b3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:31.829200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421daa0e-2380-455a-90d9-e4b213e6b3ef None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:31.829482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421daa0e-2380-455a-90d9-e4b213e6b3ef None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:31.829786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421daa0e-2380-455a-90d9-e4b213e6b3ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:31.838327 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421daa0e-2380-455a-90d9-e4b213e6b3ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:31.839779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421daa0e-2380-455a-90d9-e4b213e6b3ef None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-421daa0e-2380-455a-90d9-e4b213e6b3ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:33:31.855290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:31.856417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:31.857770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:33:31.857770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:33:31.857770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:33:31.857770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:33:31.857770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:33:31.857770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:33:31.857770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:33:31.857770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:31.857770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 555/1104] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:33:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:31.862892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab24a1-50ba-4cbb-9714-4a2627a670ef None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:31.867164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:31.979447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab24a1-50ba-4cbb-9714-4a2627a670ef None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:32.016398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa4a6a-be3a-4269-98e5-0ca92ff0dfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:32.016998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa4a6a-be3a-4269-98e5-0ca92ff0dfac None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:32.017273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa4a6a-be3a-4269-98e5-0ca92ff0dfac None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:32.017540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa4a6a-be3a-4269-98e5-0ca92ff0dfac None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:32.018259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa4a6a-be3a-4269-98e5-0ca92ff0dfac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:32.018561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa4a6a-be3a-4269-98e5-0ca92ff0dfac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:32.019989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa4a6a-be3a-4269-98e5-0ca92ff0dfac None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:32.032646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 550/1105] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:33:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:32.039439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421daa0e-2380-455a-90d9-e4b213e6b3ef None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:32.152142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421daa0e-2380-455a-90d9-e4b213e6b3ef None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:32.195287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e8c637-859a-4135-ba54-a1b762b42fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:32.195881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8c637-859a-4135-ba54-a1b762b42fd2 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:32.196155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8c637-859a-4135-ba54-a1b762b42fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:32.197041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e8c637-859a-4135-ba54-a1b762b42fd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:32.197179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8c637-859a-4135-ba54-a1b762b42fd2 None admin] RBAC: Adding query filter params (role.id=f4cfecbeb0f44d2aa4a275e60038fa7c, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=5d17aa4b23e04290b6c8b1924b0dcf32) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:32.197431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8c637-859a-4135-ba54-a1b762b42fd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:32.198999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e8c637-859a-4135-ba54-a1b762b42fd2 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:32.205746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 556/1106] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:33:32 2022] GET /identity/v3/role_assignments?user.id=5d17aa4b23e04290b6c8b1924b0dcf32&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=f4cfecbeb0f44d2aa4a275e60038fa7c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:33.542126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8329042-8719-4f2c-a9df-9d7de8ca6b87 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:33.542659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8329042-8719-4f2c-a9df-9d7de8ca6b87 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:33.542659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8329042-8719-4f2c-a9df-9d7de8ca6b87 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:33.543978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 551/1107] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:33.548979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca67bd-a4b6-4c2c-8593-5b7e6026cef7 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:33.549414 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca67bd-a4b6-4c2c-8593-5b7e6026cef7 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:33.549550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca67bd-a4b6-4c2c-8593-5b7e6026cef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:33.568202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-3aca67bd-a4b6-4c2c-8593-5b7e6026cef7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:33.569406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3aca67bd-a4b6-4c2c-8593-5b7e6026cef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:33.611496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:33.638485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 557/1108] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:33.662910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8329042-8719-4f2c-a9df-9d7de8ca6b87 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:33.729266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:33.782630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8329042-8719-4f2c-a9df-9d7de8ca6b87 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:33.818419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f55f9-2cd3-4189-a0c7-7e28e89c894f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:33.819043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f55f9-2cd3-4189-a0c7-7e28e89c894f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:33.819318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f55f9-2cd3-4189-a0c7-7e28e89c894f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:33.819585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f55f9-2cd3-4189-a0c7-7e28e89c894f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:33.820730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f55f9-2cd3-4189-a0c7-7e28e89c894f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:33.822179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55f55f9-2cd3-4189-a0c7-7e28e89c894f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:33.823771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 552/1109] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:33.830204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca67bd-a4b6-4c2c-8593-5b7e6026cef7 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:33.933863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca67bd-a4b6-4c2c-8593-5b7e6026cef7 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:33.968686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d429b6c-0bd8-47f8-a2e9-9e35a6ce356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:33.969283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d429b6c-0bd8-47f8-a2e9-9e35a6ce356a None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:33.969560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d429b6c-0bd8-47f8-a2e9-9e35a6ce356a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:33.969828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d429b6c-0bd8-47f8-a2e9-9e35a6ce356a None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:33.970509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d429b6c-0bd8-47f8-a2e9-9e35a6ce356a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:33.971934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d429b6c-0bd8-47f8-a2e9-9e35a6ce356a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:34.064836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 558/1110] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:33:33 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:33:36.121902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b873b-c65d-4f51-acc2-ac04afeed997 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:36.122487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b873b-c65d-4f51-acc2-ac04afeed997 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:36.122487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b873b-c65d-4f51-acc2-ac04afeed997 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:36.123927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 553/1111] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:36.129284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23690a97-ca94-4c98-894d-e0a6b050aa38 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:36.129729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23690a97-ca94-4c98-894d-e0a6b050aa38 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:36.129825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23690a97-ca94-4c98-894d-e0a6b050aa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:36.148092 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-23690a97-ca94-4c98-894d-e0a6b050aa38 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:36.149361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-23690a97-ca94-4c98-894d-e0a6b050aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:36.173460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:36.216007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 559/1112] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:36.239880 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b873b-c65d-4f51-acc2-ac04afeed997 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:36.340466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b873b-c65d-4f51-acc2-ac04afeed997 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:36.370517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:36.376095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7eca0d-ee3b-4a6c-82ef-cda3b7a99af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:36.376648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7eca0d-ee3b-4a6c-82ef-cda3b7a99af8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:36.376919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7eca0d-ee3b-4a6c-82ef-cda3b7a99af8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:36.377181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7eca0d-ee3b-4a6c-82ef-cda3b7a99af8 None admin] PATH_INFO: `/v3/roles/20d35b10934447938237c0db5202bbfb` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:36.378571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7eca0d-ee3b-4a6c-82ef-cda3b7a99af8 None admin] RBAC: Authorizing `identity:get_role(role_id=20d35b10934447938237c0db5202bbfb)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:36.379916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7eca0d-ee3b-4a6c-82ef-cda3b7a99af8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:36.381231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 554/1113] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:36 2022] GET /identity/v3/roles/20d35b10934447938237c0db5202bbfb => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:36.387256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23690a97-ca94-4c98-894d-e0a6b050aa38 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:36.485392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23690a97-ca94-4c98-894d-e0a6b050aa38 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:36.518715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9514e9-6f79-43d2-973e-ea51bc8e4c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:36.519335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9514e9-6f79-43d2-973e-ea51bc8e4c31 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:36.519609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9514e9-6f79-43d2-973e-ea51bc8e4c31 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:36.519875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9514e9-6f79-43d2-973e-ea51bc8e4c31 None admin] PATH_INFO: `/v3/users/b7b5b37bd96246549aec635c73017fe5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:36.533601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9514e9-6f79-43d2-973e-ea51bc8e4c31 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b5b37bd96246549aec635c73017fe5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:36.535117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9514e9-6f79-43d2-973e-ea51bc8e4c31 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:36.536788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 560/1114] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:36 2022] GET /identity/v3/users/b7b5b37bd96246549aec635c73017fe5 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:36.543186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7eca0d-ee3b-4a6c-82ef-cda3b7a99af8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:36.641869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7eca0d-ee3b-4a6c-82ef-cda3b7a99af8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:36.675058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704e4a6-b74f-404a-b582-ef183ad0e921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:36.675650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704e4a6-b74f-404a-b582-ef183ad0e921 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:36.675924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704e4a6-b74f-404a-b582-ef183ad0e921 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:36.676190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704e4a6-b74f-404a-b582-ef183ad0e921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:36.799050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:36.802668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704e4a6-b74f-404a-b582-ef183ad0e921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:36.803955 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704e4a6-b74f-404a-b582-ef183ad0e921 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:36.987517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 555/1115] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:33:36.993440 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9514e9-6f79-43d2-973e-ea51bc8e4c31 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:37.018226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:37.101825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9514e9-6f79-43d2-973e-ea51bc8e4c31 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:37.136777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d654ab-dba1-42a5-8c3b-8d5d0671dfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:37.137457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d654ab-dba1-42a5-8c3b-8d5d0671dfe7 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:37.137654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d654ab-dba1-42a5-8c3b-8d5d0671dfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:37.137920 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d654ab-dba1-42a5-8c3b-8d5d0671dfe7 None admin] PATH_INFO: `/v3/projects/73b27f49aa19494f85cd53089a1e5224` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:37.139134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d654ab-dba1-42a5-8c3b-8d5d0671dfe7 None admin] RBAC: Authorizing `identity:get_project(project_id=73b27f49aa19494f85cd53089a1e5224)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:37.140549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d654ab-dba1-42a5-8c3b-8d5d0671dfe7 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:37.142199 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 561/1116] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:33:36 2022] GET /identity/v3/projects/73b27f49aa19494f85cd53089a1e5224 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:37.149261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704e4a6-b74f-404a-b582-ef183ad0e921 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:37.246201 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:37.250119 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704e4a6-b74f-404a-b582-ef183ad0e921 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:37.285652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ebd5b1-7185-455a-bde4-f71f83a3d88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:37.286313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebd5b1-7185-455a-bde4-f71f83a3d88b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:37.286623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebd5b1-7185-455a-bde4-f71f83a3d88b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:37.286901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ebd5b1-7185-455a-bde4-f71f83a3d88b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:37.287669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebd5b1-7185-455a-bde4-f71f83a3d88b None admin] RBAC: Adding query filter params (role.id=20d35b10934447938237c0db5202bbfb, scope.project.id=73b27f49aa19494f85cd53089a1e5224, user.id=b7b5b37bd96246549aec635c73017fe5) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:37.287997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebd5b1-7185-455a-bde4-f71f83a3d88b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:37.289465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ebd5b1-7185-455a-bde4-f71f83a3d88b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:37.294778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 556/1117] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:33:37 2022] GET /identity/v3/role_assignments?user.id=b7b5b37bd96246549aec635c73017fe5&scope.project.id=73b27f49aa19494f85cd53089a1e5224&role.id=20d35b10934447938237c0db5202bbfb => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:39.298707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f95fd-461c-4630-a4f5-d3ce0a3cb893 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:39.299252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f95fd-461c-4630-a4f5-d3ce0a3cb893 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:39.299252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f95fd-461c-4630-a4f5-d3ce0a3cb893 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:39.300562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 562/1118] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:39.305915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152f52c-1d95-4a17-9cf9-0c44bce1c063 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:39.306399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152f52c-1d95-4a17-9cf9-0c44bce1c063 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:39.306499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152f52c-1d95-4a17-9cf9-0c44bce1c063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:39.325145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f152f52c-1d95-4a17-9cf9-0c44bce1c063 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:39.326476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f152f52c-1d95-4a17-9cf9-0c44bce1c063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:39.393735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 557/1119] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:39.417039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f95fd-461c-4630-a4f5-d3ce0a3cb893 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:39.500297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:39.515579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f95fd-461c-4630-a4f5-d3ce0a3cb893 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:39.549160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d777e-5ce8-4e74-836b-cfae149e2da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:39.549700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d777e-5ce8-4e74-836b-cfae149e2da6 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:39.549992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d777e-5ce8-4e74-836b-cfae149e2da6 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:39.550260 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d777e-5ce8-4e74-836b-cfae149e2da6 None admin] PATH_INFO: `/v3/roles/20d35b10934447938237c0db5202bbfb` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:39.551609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d777e-5ce8-4e74-836b-cfae149e2da6 None admin] RBAC: Authorizing `identity:get_role(role_id=20d35b10934447938237c0db5202bbfb)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:39.552957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d777e-5ce8-4e74-836b-cfae149e2da6 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:39.554327 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 563/1120] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:39 2022] GET /identity/v3/roles/20d35b10934447938237c0db5202bbfb => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:39.560132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152f52c-1d95-4a17-9cf9-0c44bce1c063 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:39.649905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:39.658299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152f52c-1d95-4a17-9cf9-0c44bce1c063 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:39.691213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7f15f-088d-49f0-a1c4-9411f23c9798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:39.691821 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7f15f-088d-49f0-a1c4-9411f23c9798 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:39.692096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7f15f-088d-49f0-a1c4-9411f23c9798 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:39.692363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7f15f-088d-49f0-a1c4-9411f23c9798 None admin] PATH_INFO: `/v3/users/b7b5b37bd96246549aec635c73017fe5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:39.693462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7f15f-088d-49f0-a1c4-9411f23c9798 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b5b37bd96246549aec635c73017fe5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:39.695540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7f15f-088d-49f0-a1c4-9411f23c9798 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:39.697099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 558/1121] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:39 2022] GET /identity/v3/users/b7b5b37bd96246549aec635c73017fe5 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:39.702949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d777e-5ce8-4e74-836b-cfae149e2da6 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:39.795975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d777e-5ce8-4e74-836b-cfae149e2da6 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:39.828476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b9d96-cbbc-403f-982c-a6d3e15494f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:39.829009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b9d96-cbbc-403f-982c-a6d3e15494f2 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:39.829277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b9d96-cbbc-403f-982c-a6d3e15494f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:39.829538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b9d96-cbbc-403f-982c-a6d3e15494f2 None admin] PATH_INFO: `/v3/projects/73b27f49aa19494f85cd53089a1e5224` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:39.830633 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b9d96-cbbc-403f-982c-a6d3e15494f2 None admin] RBAC: Authorizing `identity:get_project(project_id=73b27f49aa19494f85cd53089a1e5224)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:39.832025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b9d96-cbbc-403f-982c-a6d3e15494f2 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:39.833605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 564/1122] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:33:39 2022] GET /identity/v3/projects/73b27f49aa19494f85cd53089a1e5224 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:39.839699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7f15f-088d-49f0-a1c4-9411f23c9798 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:39.937148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7f15f-088d-49f0-a1c4-9411f23c9798 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:39.973392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cd17f-8fec-4a02-b0ea-0ee4cbd537f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:39.974079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cd17f-8fec-4a02-b0ea-0ee4cbd537f3 None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:39.974360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cd17f-8fec-4a02-b0ea-0ee4cbd537f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:39.974631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cd17f-8fec-4a02-b0ea-0ee4cbd537f3 None admin] PATH_INFO: `/v3/projects/73b27f49aa19494f85cd53089a1e5224/users/b7b5b37bd96246549aec635c73017fe5/roles/20d35b10934447938237c0db5202bbfb` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:39.977062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cd17f-8fec-4a02-b0ea-0ee4cbd537f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=73b27f49aa19494f85cd53089a1e5224, user_id=b7b5b37bd96246549aec635c73017fe5, role_id=20d35b10934447938237c0db5202bbfb)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:39.978525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cd17f-8fec-4a02-b0ea-0ee4cbd537f3 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:40.082037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 559/1123] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:33:39 2022] PUT /identity/v3/projects/73b27f49aa19494f85cd53089a1e5224/users/b7b5b37bd96246549aec635c73017fe5/roles/20d35b10934447938237c0db5202bbfb => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:33:42.086214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391251a-6c79-454d-83a2-4a9eb2ce87c2 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:42.086749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391251a-6c79-454d-83a2-4a9eb2ce87c2 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:42.086749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391251a-6c79-454d-83a2-4a9eb2ce87c2 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:42.088055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 565/1124] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:42.094441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7194e6-c223-4960-9114-ab5ba90357a5 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:42.094891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7194e6-c223-4960-9114-ab5ba90357a5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:42.094986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7194e6-c223-4960-9114-ab5ba90357a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:42.112858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6a7194e6-c223-4960-9114-ab5ba90357a5 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:42.114077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6a7194e6-c223-4960-9114-ab5ba90357a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:42.128405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:42.180200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 560/1125] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:42.203422 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391251a-6c79-454d-83a2-4a9eb2ce87c2 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:42.226872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:42.300093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391251a-6c79-454d-83a2-4a9eb2ce87c2 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:42.332116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424264c-a993-4784-8c9b-54fa2184b6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:42.332717 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424264c-a993-4784-8c9b-54fa2184b6af None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:42.332991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424264c-a993-4784-8c9b-54fa2184b6af None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:42.333252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424264c-a993-4784-8c9b-54fa2184b6af None admin] PATH_INFO: `/v3/roles/20d35b10934447938237c0db5202bbfb` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:42.334625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424264c-a993-4784-8c9b-54fa2184b6af None admin] RBAC: Authorizing `identity:get_role(role_id=20d35b10934447938237c0db5202bbfb)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:42.335989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424264c-a993-4784-8c9b-54fa2184b6af None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:42.337333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 566/1126] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:42 2022] GET /identity/v3/roles/20d35b10934447938237c0db5202bbfb => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:42.343458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7194e6-c223-4960-9114-ab5ba90357a5 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:42.440564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7194e6-c223-4960-9114-ab5ba90357a5 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:42.472925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f4612-3514-4daf-b415-c45e75df9721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:42.473535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f4612-3514-4daf-b415-c45e75df9721 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:42.473811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f4612-3514-4daf-b415-c45e75df9721 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:42.474110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f4612-3514-4daf-b415-c45e75df9721 None admin] PATH_INFO: `/v3/users/b7b5b37bd96246549aec635c73017fe5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:42.475204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f4612-3514-4daf-b415-c45e75df9721 None admin] RBAC: Authorizing `identity:get_user(user_id=b7b5b37bd96246549aec635c73017fe5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:42.477133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f4612-3514-4daf-b415-c45e75df9721 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:42.478718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 561/1127] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:42 2022] GET /identity/v3/users/b7b5b37bd96246549aec635c73017fe5 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:42.484623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424264c-a993-4784-8c9b-54fa2184b6af None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:42.574473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:42.583252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424264c-a993-4784-8c9b-54fa2184b6af None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:42.617977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd77141-018f-41a2-8fe2-e1cb05920612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:42.618507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd77141-018f-41a2-8fe2-e1cb05920612 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:42.618779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd77141-018f-41a2-8fe2-e1cb05920612 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:42.619044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd77141-018f-41a2-8fe2-e1cb05920612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:42.682605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd77141-018f-41a2-8fe2-e1cb05920612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:42.684035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd77141-018f-41a2-8fe2-e1cb05920612 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:42.767326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:42.777552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 567/1128] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:33:42.783511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f4612-3514-4daf-b415-c45e75df9721 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:42.802372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:42.881203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f4612-3514-4daf-b415-c45e75df9721 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:42.915850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd32eba-e834-44b6-becd-1ecc4523cd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:42.916481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd32eba-e834-44b6-becd-1ecc4523cd0a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:42.916755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd32eba-e834-44b6-becd-1ecc4523cd0a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:42.917014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd32eba-e834-44b6-becd-1ecc4523cd0a None admin] PATH_INFO: `/v3/projects/73b27f49aa19494f85cd53089a1e5224` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:42.918151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd32eba-e834-44b6-becd-1ecc4523cd0a None admin] RBAC: Authorizing `identity:get_project(project_id=73b27f49aa19494f85cd53089a1e5224)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:42.919572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd32eba-e834-44b6-becd-1ecc4523cd0a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:42.921197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 562/1129] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:33:42 2022] GET /identity/v3/projects/73b27f49aa19494f85cd53089a1e5224 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:42.927494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd77141-018f-41a2-8fe2-e1cb05920612 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:43.024615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd77141-018f-41a2-8fe2-e1cb05920612 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:43.059482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e23984-db8d-4fc1-ae28-acaedc62ec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:43.060065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e23984-db8d-4fc1-ae28-acaedc62ec69 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:43.060335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e23984-db8d-4fc1-ae28-acaedc62ec69 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:43.060595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e23984-db8d-4fc1-ae28-acaedc62ec69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:43.061299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e23984-db8d-4fc1-ae28-acaedc62ec69 None admin] RBAC: Adding query filter params (role.id=20d35b10934447938237c0db5202bbfb, scope.project.id=73b27f49aa19494f85cd53089a1e5224, user.id=b7b5b37bd96246549aec635c73017fe5) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:43.061595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e23984-db8d-4fc1-ae28-acaedc62ec69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:43.063027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e23984-db8d-4fc1-ae28-acaedc62ec69 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:43.068575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 568/1130] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:33:42 2022] GET /identity/v3/role_assignments?user.id=b7b5b37bd96246549aec635c73017fe5&scope.project.id=73b27f49aa19494f85cd53089a1e5224&role.id=20d35b10934447938237c0db5202bbfb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:45.059526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91a38-fe70-43a6-9cb4-f8fd0c4adc79 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:45.060170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91a38-fe70-43a6-9cb4-f8fd0c4adc79 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:45.060170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b91a38-fe70-43a6-9cb4-f8fd0c4adc79 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:45.061354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 563/1131] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:45.066609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe584b-4958-4e65-986a-708704fdb87b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:45.067029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe584b-4958-4e65-986a-708704fdb87b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:45.067158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe584b-4958-4e65-986a-708704fdb87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:45.085368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-1ebe584b-4958-4e65-986a-708704fdb87b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:45.086603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1ebe584b-4958-4e65-986a-708704fdb87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:45.137091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:45.154955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 569/1132] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:45.178424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91a38-fe70-43a6-9cb4-f8fd0c4adc79 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:45.260945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:45.276362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b91a38-fe70-43a6-9cb4-f8fd0c4adc79 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:45.309592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444e9e6-5642-4b46-9635-220cf89444fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:45.310154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444e9e6-5642-4b46-9635-220cf89444fd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:45.310434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444e9e6-5642-4b46-9635-220cf89444fd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:45.310696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444e9e6-5642-4b46-9635-220cf89444fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:45.311797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444e9e6-5642-4b46-9635-220cf89444fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:45.313198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444e9e6-5642-4b46-9635-220cf89444fd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:45.314770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 564/1133] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:45.321114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe584b-4958-4e65-986a-708704fdb87b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:45.414111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe584b-4958-4e65-986a-708704fdb87b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:45.445858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54584b25-457e-4e6a-ac0e-9f81f240614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:45.446466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54584b25-457e-4e6a-ac0e-9f81f240614b None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:45.446736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54584b25-457e-4e6a-ac0e-9f81f240614b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:45.447000 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54584b25-457e-4e6a-ac0e-9f81f240614b None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:45.447631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54584b25-457e-4e6a-ac0e-9f81f240614b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:45.449031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54584b25-457e-4e6a-ac0e-9f81f240614b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:45.562724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 570/1134] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:33:45 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:33:47.578516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76462da-7f33-4c9d-a398-8f88ea8b2aa8 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:47.579055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76462da-7f33-4c9d-a398-8f88ea8b2aa8 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:47.579055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76462da-7f33-4c9d-a398-8f88ea8b2aa8 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:47.580351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 565/1135] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:47.585508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426e2a5-870a-46da-9d01-aae0a6544af3 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:47.585972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426e2a5-870a-46da-9d01-aae0a6544af3 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:47.586068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9426e2a5-870a-46da-9d01-aae0a6544af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:47.604477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9426e2a5-870a-46da-9d01-aae0a6544af3 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:47.605657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9426e2a5-870a-46da-9d01-aae0a6544af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:47.634083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:47.671735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 571/1136] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:47.696817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76462da-7f33-4c9d-a398-8f88ea8b2aa8 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:47.798493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76462da-7f33-4c9d-a398-8f88ea8b2aa8 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:47.831226 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0d82b-531f-48b4-b8da-6df346c962cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:47.831831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0d82b-531f-48b4-b8da-6df346c962cc None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:47.832104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0d82b-531f-48b4-b8da-6df346c962cc None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:47.832367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0d82b-531f-48b4-b8da-6df346c962cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:47.833436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0d82b-531f-48b4-b8da-6df346c962cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:47.834870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0d82b-531f-48b4-b8da-6df346c962cc None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:47.836402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 566/1137] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:33:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:47.842483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426e2a5-870a-46da-9d01-aae0a6544af3 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:47.936363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9426e2a5-870a-46da-9d01-aae0a6544af3 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:47.969467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb326dcb-0740-406e-b791-257a4ee652ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:47.970079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb326dcb-0740-406e-b791-257a4ee652ce None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:47.970350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb326dcb-0740-406e-b791-257a4ee652ce None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:47.970614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb326dcb-0740-406e-b791-257a4ee652ce None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:47.971237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb326dcb-0740-406e-b791-257a4ee652ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:47.972638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb326dcb-0740-406e-b791-257a4ee652ce None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:48.083237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 572/1138] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:33:47 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:33:50.135333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb588d8-3b34-403a-ae7a-f3f41378993c None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:50.135871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb588d8-3b34-403a-ae7a-f3f41378993c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:50.135871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb588d8-3b34-403a-ae7a-f3f41378993c None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:50.137160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 567/1139] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:50.142860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfa10-a89d-476b-9db8-21318e7d745b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:50.143363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfa10-a89d-476b-9db8-21318e7d745b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:50.143457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebfa10-a89d-476b-9db8-21318e7d745b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:50.162292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-b2ebfa10-a89d-476b-9db8-21318e7d745b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:50.163527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b2ebfa10-a89d-476b-9db8-21318e7d745b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:50.181400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:50.237317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 573/1140] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:50.261493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb588d8-3b34-403a-ae7a-f3f41378993c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:50.288192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:50.367809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb588d8-3b34-403a-ae7a-f3f41378993c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:50.401705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af23529-7e78-4a99-a7fa-2d85ff16443f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:50.402278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af23529-7e78-4a99-a7fa-2d85ff16443f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:50.402553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af23529-7e78-4a99-a7fa-2d85ff16443f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:50.402819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af23529-7e78-4a99-a7fa-2d85ff16443f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:50.412966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af23529-7e78-4a99-a7fa-2d85ff16443f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-2af23529-7e78-4a99-a7fa-2d85ff16443f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:50.417504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:50.418741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:50.420136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 568/1141] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:50.424963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebfa10-a89d-476b-9db8-21318e7d745b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:50.533217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebfa10-a89d-476b-9db8-21318e7d745b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:50.568948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e2d38-5f18-4414-942d-b4773a5b998f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:50.569548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e2d38-5f18-4414-942d-b4773a5b998f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:50.569826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e2d38-5f18-4414-942d-b4773a5b998f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:50.570126 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e2d38-5f18-4414-942d-b4773a5b998f None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:50.570789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e2d38-5f18-4414-942d-b4773a5b998f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:50.571095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e2d38-5f18-4414-942d-b4773a5b998f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:50.572454 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e2d38-5f18-4414-942d-b4773a5b998f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:50.580871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 574/1142] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:33:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:50.586909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af23529-7e78-4a99-a7fa-2d85ff16443f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:50.671386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:50.690865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af23529-7e78-4a99-a7fa-2d85ff16443f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:50.725864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25275c-b931-4ba1-b58b-e45bb6708610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:50.726535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25275c-b931-4ba1-b58b-e45bb6708610 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:50.726838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25275c-b931-4ba1-b58b-e45bb6708610 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:50.727107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25275c-b931-4ba1-b58b-e45bb6708610 None admin] PATH_INFO: `/v3/users/e6c6a8a6293e4ab6b7cba4e01256a34c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:50.741482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25275c-b931-4ba1-b58b-e45bb6708610 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c6a8a6293e4ab6b7cba4e01256a34c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:50.742976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a25275c-b931-4ba1-b58b-e45bb6708610 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:50.744652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 569/1143] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:50 2022] GET /identity/v3/users/e6c6a8a6293e4ab6b7cba4e01256a34c => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:50.750452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e2d38-5f18-4414-942d-b4773a5b998f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:50.766847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:50.857333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e2d38-5f18-4414-942d-b4773a5b998f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:50.890923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5156c-aaaa-4727-a202-c57118ceed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:50.891519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5156c-aaaa-4727-a202-c57118ceed3a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:50.891807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5156c-aaaa-4727-a202-c57118ceed3a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:50.892059 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5156c-aaaa-4727-a202-c57118ceed3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:50.959459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5156c-aaaa-4727-a202-c57118ceed3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:50.961334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5156c-aaaa-4727-a202-c57118ceed3a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:50.976659 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:51.065008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 575/1144] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:33:51.070976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25275c-b931-4ba1-b58b-e45bb6708610 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:51.179609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25275c-b931-4ba1-b58b-e45bb6708610 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:51.217042 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43afa6-2910-4621-ad01-e0b2e9e091cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:51.217583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43afa6-2910-4621-ad01-e0b2e9e091cd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:51.217864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43afa6-2910-4621-ad01-e0b2e9e091cd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:51.218166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43afa6-2910-4621-ad01-e0b2e9e091cd None admin] PATH_INFO: `/v3/projects/c3fde240017845b8993cda5c5ed144f5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:51.219294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43afa6-2910-4621-ad01-e0b2e9e091cd None admin] RBAC: Authorizing `identity:get_project(project_id=c3fde240017845b8993cda5c5ed144f5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:51.220709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43afa6-2910-4621-ad01-e0b2e9e091cd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:51.222389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 570/1145] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:33:51 2022] GET /identity/v3/projects/c3fde240017845b8993cda5c5ed144f5 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:51.229175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5156c-aaaa-4727-a202-c57118ceed3a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:51.330741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5156c-aaaa-4727-a202-c57118ceed3a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:51.345442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:51.364764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551ed79-ae00-44ff-a24b-0373ec494a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:51.365360 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551ed79-ae00-44ff-a24b-0373ec494a9a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:51.365631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551ed79-ae00-44ff-a24b-0373ec494a9a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:51.365894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551ed79-ae00-44ff-a24b-0373ec494a9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:51.366632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551ed79-ae00-44ff-a24b-0373ec494a9a None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=c3fde240017845b8993cda5c5ed144f5, user.id=e6c6a8a6293e4ab6b7cba4e01256a34c) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:51.366928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551ed79-ae00-44ff-a24b-0373ec494a9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:51.368333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551ed79-ae00-44ff-a24b-0373ec494a9a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:51.374093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 576/1146] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:33:51 2022] GET /identity/v3/role_assignments?user.id=e6c6a8a6293e4ab6b7cba4e01256a34c&scope.project.id=c3fde240017845b8993cda5c5ed144f5&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:53.347438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe78e8a-2bee-4fa5-9a30-a0a95b12d7f6 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:53.348012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe78e8a-2bee-4fa5-9a30-a0a95b12d7f6 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:53.348012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe78e8a-2bee-4fa5-9a30-a0a95b12d7f6 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:53.349408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 571/1147] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:53.354816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf7aca-cba9-4c72-bdbe-57eb69f2fc55 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:53.355248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf7aca-cba9-4c72-bdbe-57eb69f2fc55 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:53.355358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf7aca-cba9-4c72-bdbe-57eb69f2fc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:53.374718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9dcf7aca-cba9-4c72-bdbe-57eb69f2fc55 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:53.375909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9dcf7aca-cba9-4c72-bdbe-57eb69f2fc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:53.443716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 577/1148] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:53.465798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe78e8a-2bee-4fa5-9a30-a0a95b12d7f6 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:53.476893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:53.575999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe78e8a-2bee-4fa5-9a30-a0a95b12d7f6 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:53.609658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f9ce8-32ab-4e21-a5f6-be414d0ab4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:53.610224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f9ce8-32ab-4e21-a5f6-be414d0ab4c0 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:53.610498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f9ce8-32ab-4e21-a5f6-be414d0ab4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:53.610765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f9ce8-32ab-4e21-a5f6-be414d0ab4c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:53.621147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f9ce8-32ab-4e21-a5f6-be414d0ab4c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-6b5f9ce8-32ab-4e21-a5f6-be414d0ab4c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:53.625735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:53.626896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:33:53.628161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 572/1149] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:33:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:53.632828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf7aca-cba9-4c72-bdbe-57eb69f2fc55 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:53.731322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf7aca-cba9-4c72-bdbe-57eb69f2fc55 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:53.740146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:53.764063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dada5ca-6d4c-45be-be61-eaa230166fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:53.764651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dada5ca-6d4c-45be-be61-eaa230166fa7 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:53.764930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dada5ca-6d4c-45be-be61-eaa230166fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:53.765193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dada5ca-6d4c-45be-be61-eaa230166fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:53.765846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dada5ca-6d4c-45be-be61-eaa230166fa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:53.766178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dada5ca-6d4c-45be-be61-eaa230166fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:53.767531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dada5ca-6d4c-45be-be61-eaa230166fa7 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:53.775712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 578/1150] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:33:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:53.781506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f9ce8-32ab-4e21-a5f6-be414d0ab4c0 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:53.845819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:53.878689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f9ce8-32ab-4e21-a5f6-be414d0ab4c0 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:53.911069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c0018-cd9b-485c-a6a9-167979d51472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:53.911666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c0018-cd9b-485c-a6a9-167979d51472 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:53.911938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c0018-cd9b-485c-a6a9-167979d51472 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:53.912204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c0018-cd9b-485c-a6a9-167979d51472 None admin] PATH_INFO: `/v3/users/e6c6a8a6293e4ab6b7cba4e01256a34c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:53.913791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c0018-cd9b-485c-a6a9-167979d51472 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c6a8a6293e4ab6b7cba4e01256a34c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:53.915221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c0018-cd9b-485c-a6a9-167979d51472 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:53.916764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 573/1151] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:53 2022] GET /identity/v3/users/e6c6a8a6293e4ab6b7cba4e01256a34c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:53.922623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dada5ca-6d4c-45be-be61-eaa230166fa7 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:54.018996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dada5ca-6d4c-45be-be61-eaa230166fa7 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:54.052435 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f1243-51d1-48de-bc90-750753552bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:54.053041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f1243-51d1-48de-bc90-750753552bf2 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:54.053317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f1243-51d1-48de-bc90-750753552bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:54.053583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f1243-51d1-48de-bc90-750753552bf2 None admin] PATH_INFO: `/v3/projects/c3fde240017845b8993cda5c5ed144f5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:54.055289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f1243-51d1-48de-bc90-750753552bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=c3fde240017845b8993cda5c5ed144f5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:54.056707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f1243-51d1-48de-bc90-750753552bf2 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:54.058349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 579/1152] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:33:53 2022] GET /identity/v3/projects/c3fde240017845b8993cda5c5ed144f5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:54.064246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c0018-cd9b-485c-a6a9-167979d51472 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:54.161733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c0018-cd9b-485c-a6a9-167979d51472 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:54.196004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174c5fc-beec-420e-a34e-5ddb7f534641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:54.196643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c5fc-beec-420e-a34e-5ddb7f534641 None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:54.196914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c5fc-beec-420e-a34e-5ddb7f534641 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:54.197177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c5fc-beec-420e-a34e-5ddb7f534641 None admin] PATH_INFO: `/v3/projects/c3fde240017845b8993cda5c5ed144f5/users/e6c6a8a6293e4ab6b7cba4e01256a34c/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:54.199279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174c5fc-beec-420e-a34e-5ddb7f534641 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3fde240017845b8993cda5c5ed144f5, user_id=e6c6a8a6293e4ab6b7cba4e01256a34c, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:54.200940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174c5fc-beec-420e-a34e-5ddb7f534641 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:54.204056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:54.301572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 574/1153] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:33:54 2022] PUT /identity/v3/projects/c3fde240017845b8993cda5c5ed144f5/users/e6c6a8a6293e4ab6b7cba4e01256a34c/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:33:56.305974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc7b45-7a8e-4d75-a49c-1f0a8620e6e5 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:56.306540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc7b45-7a8e-4d75-a49c-1f0a8620e6e5 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:56.306540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc7b45-7a8e-4d75-a49c-1f0a8620e6e5 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:56.307854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 580/1154] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:56.312928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d7ac-c19a-4a5c-8cdc-d2a975691317 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:56.313351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d7ac-c19a-4a5c-8cdc-d2a975691317 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:56.313443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db8d7ac-c19a-4a5c-8cdc-d2a975691317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:56.331372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-4db8d7ac-c19a-4a5c-8cdc-d2a975691317 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:56.332547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4db8d7ac-c19a-4a5c-8cdc-d2a975691317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:56.397934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 575/1155] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:33:56.421399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc7b45-7a8e-4d75-a49c-1f0a8620e6e5 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:56.462205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:56.530656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc7b45-7a8e-4d75-a49c-1f0a8620e6e5 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:56.574884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e78ce-129c-4043-8254-b0402b95f409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:56.575823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e78ce-129c-4043-8254-b0402b95f409 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:56.576305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e78ce-129c-4043-8254-b0402b95f409 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:56.576774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e78ce-129c-4043-8254-b0402b95f409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:56.590101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e78ce-129c-4043-8254-b0402b95f409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-7e1e78ce-129c-4043-8254-b0402b95f409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:33:56.594980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:33:56.596130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:33:56.597015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 581/1156] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:33:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:33:56.602266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8d7ac-c19a-4a5c-8cdc-d2a975691317 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:56.706856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db8d7ac-c19a-4a5c-8cdc-d2a975691317 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:56.740735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:56.752056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1208f2f-b1e2-4858-ae1b-003b12f7973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:56.752973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1208f2f-b1e2-4858-ae1b-003b12f7973e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:56.753395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1208f2f-b1e2-4858-ae1b-003b12f7973e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:56.753662 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1208f2f-b1e2-4858-ae1b-003b12f7973e None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:56.754495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1208f2f-b1e2-4858-ae1b-003b12f7973e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:56.754999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1208f2f-b1e2-4858-ae1b-003b12f7973e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:56.756866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1208f2f-b1e2-4858-ae1b-003b12f7973e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:56.766841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 576/1157] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:33:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:56.773487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e78ce-129c-4043-8254-b0402b95f409 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:56.877972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:56.881754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e78ce-129c-4043-8254-b0402b95f409 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:56.919023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7b967-1b68-45df-a4f2-41313ba46545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:56.919638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7b967-1b68-45df-a4f2-41313ba46545 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:56.919911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7b967-1b68-45df-a4f2-41313ba46545 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:56.920172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7b967-1b68-45df-a4f2-41313ba46545 None admin] PATH_INFO: `/v3/users/e6c6a8a6293e4ab6b7cba4e01256a34c` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:56.921313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7b967-1b68-45df-a4f2-41313ba46545 None admin] RBAC: Authorizing `identity:get_user(user_id=e6c6a8a6293e4ab6b7cba4e01256a34c)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:56.922768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7b967-1b68-45df-a4f2-41313ba46545 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:56.924372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 582/1158] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:33:56 2022] GET /identity/v3/users/e6c6a8a6293e4ab6b7cba4e01256a34c => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:33:56.931067 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1208f2f-b1e2-4858-ae1b-003b12f7973e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:57.032608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1208f2f-b1e2-4858-ae1b-003b12f7973e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:57.067346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e7ada-bbe0-4486-88b2-9391a59de5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:57.067949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e7ada-bbe0-4486-88b2-9391a59de5bd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:57.068224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e7ada-bbe0-4486-88b2-9391a59de5bd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:57.068490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2e7ada-bbe0-4486-88b2-9391a59de5bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:57.128196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:57.140881 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e7ada-bbe0-4486-88b2-9391a59de5bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:57.142358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2e7ada-bbe0-4486-88b2-9391a59de5bd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:57.242547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 577/1159] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:33:57.249010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7b967-1b68-45df-a4f2-41313ba46545 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:57.348934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7b967-1b68-45df-a4f2-41313ba46545 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:57.382673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15492ed0-d430-48e7-a657-c016bb11a63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:57.383192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15492ed0-d430-48e7-a657-c016bb11a63d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:57.383462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15492ed0-d430-48e7-a657-c016bb11a63d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:57.383726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15492ed0-d430-48e7-a657-c016bb11a63d None admin] PATH_INFO: `/v3/projects/c3fde240017845b8993cda5c5ed144f5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:57.384828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15492ed0-d430-48e7-a657-c016bb11a63d None admin] RBAC: Authorizing `identity:get_project(project_id=c3fde240017845b8993cda5c5ed144f5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:57.386280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15492ed0-d430-48e7-a657-c016bb11a63d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:57.387903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 583/1160] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:33:57 2022] GET /identity/v3/projects/c3fde240017845b8993cda5c5ed144f5 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:57.394440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e7ada-bbe0-4486-88b2-9391a59de5bd None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:57.482576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:57.498185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2e7ada-bbe0-4486-88b2-9391a59de5bd None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:57.541523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54444a9e-e038-4553-88d9-d859f0bff00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:57.542162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54444a9e-e038-4553-88d9-d859f0bff00c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:57.542497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54444a9e-e038-4553-88d9-d859f0bff00c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:57.542765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54444a9e-e038-4553-88d9-d859f0bff00c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:57.543514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54444a9e-e038-4553-88d9-d859f0bff00c None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=c3fde240017845b8993cda5c5ed144f5, user.id=e6c6a8a6293e4ab6b7cba4e01256a34c) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:33:57.543813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54444a9e-e038-4553-88d9-d859f0bff00c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:57.545229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54444a9e-e038-4553-88d9-d859f0bff00c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:57.551137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 578/1161] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:33:57 2022] GET /identity/v3/role_assignments?user.id=e6c6a8a6293e4ab6b7cba4e01256a34c&scope.project.id=c3fde240017845b8993cda5c5ed144f5&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:33:59.610893 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda1a2a-b222-46c4-9f23-d20b90b5350b None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:59.611682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda1a2a-b222-46c4-9f23-d20b90b5350b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:59.611682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda1a2a-b222-46c4-9f23-d20b90b5350b None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:59.612873 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 584/1162] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:33:59.618748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2a1b7-32d8-4378-84e6-452ca150f899 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:59.619232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2a1b7-32d8-4378-84e6-452ca150f899 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:59.619343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2a1b7-32d8-4378-84e6-452ca150f899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:59.637911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b1d2a1b7-32d8-4378-84e6-452ca150f899 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:33:59.639165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b1d2a1b7-32d8-4378-84e6-452ca150f899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:33:59.723302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 579/1163] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:33:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:33:59.745894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda1a2a-b222-46c4-9f23-d20b90b5350b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:33:59.782384 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:33:59.848114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda1a2a-b222-46c4-9f23-d20b90b5350b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:33:59.880103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202a752-9859-4442-a4ae-0e234c971631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:33:59.880703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202a752-9859-4442-a4ae-0e234c971631 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:33:59.880974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202a752-9859-4442-a4ae-0e234c971631 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:33:59.881235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202a752-9859-4442-a4ae-0e234c971631 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:33:59.947549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202a752-9859-4442-a4ae-0e234c971631 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:33:59.948998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202a752-9859-4442-a4ae-0e234c971631 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:33:59.980616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:00.045613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 585/1164] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:33:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:34:00.051469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2a1b7-32d8-4378-84e6-452ca150f899 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:00.131557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:00.149545 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2a1b7-32d8-4378-84e6-452ca150f899 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:00.182922 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6838b-80c6-447a-8d1f-b6e3254d26f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:00.183541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6838b-80c6-447a-8d1f-b6e3254d26f2 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:00.183816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6838b-80c6-447a-8d1f-b6e3254d26f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:00.184085 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6838b-80c6-447a-8d1f-b6e3254d26f2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:00.190752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6838b-80c6-447a-8d1f-b6e3254d26f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:00.192180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6838b-80c6-447a-8d1f-b6e3254d26f2 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-38d6838b-80c6-447a-8d1f-b6e3254d26f2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:34:00.204648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:34:00.205912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:34:00.206929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 580/1165] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Aug 9 16:34:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:34:00.211916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202a752-9859-4442-a4ae-0e234c971631 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:00.312506 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202a752-9859-4442-a4ae-0e234c971631 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:00.347236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783271f9-93f1-47ae-b397-2fcaec279114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:00.347823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783271f9-93f1-47ae-b397-2fcaec279114 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:00.348094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783271f9-93f1-47ae-b397-2fcaec279114 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:00.348357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783271f9-93f1-47ae-b397-2fcaec279114 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:00.348990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783271f9-93f1-47ae-b397-2fcaec279114 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:34:00.349283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783271f9-93f1-47ae-b397-2fcaec279114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:00.350662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783271f9-93f1-47ae-b397-2fcaec279114 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:00.352544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:00.358647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 586/1166] 10.222.0.79 () {56 vars in 1115 bytes} [Tue Aug 9 16:34:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:02.339581 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6837f8f-8189-4e69-89c7-c8775589b938 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:02.340143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6837f8f-8189-4e69-89c7-c8775589b938 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:02.340143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6837f8f-8189-4e69-89c7-c8775589b938 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:02.341800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 581/1167] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:34:02.346913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28bd8-dd6f-43f7-a392-9e18d106972e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:02.347355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28bd8-dd6f-43f7-a392-9e18d106972e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:02.347447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb28bd8-dd6f-43f7-a392-9e18d106972e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:02.365201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-3eb28bd8-dd6f-43f7-a392-9e18d106972e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:34:02.366904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3eb28bd8-dd6f-43f7-a392-9e18d106972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:34:02.430899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 587/1168] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:34:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:34:02.453837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6837f8f-8189-4e69-89c7-c8775589b938 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:02.557021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6837f8f-8189-4e69-89c7-c8775589b938 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:02.589780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49963f08-0d71-4718-8181-d76d64409a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:02.590330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49963f08-0d71-4718-8181-d76d64409a81 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:02.590601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49963f08-0d71-4718-8181-d76d64409a81 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:02.590864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49963f08-0d71-4718-8181-d76d64409a81 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:02.591771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49963f08-0d71-4718-8181-d76d64409a81 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:02.593112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49963f08-0d71-4718-8181-d76d64409a81 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:02.670812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 582/1169] 10.222.0.79 () {60 vars in 1138 bytes} [Tue Aug 9 16:34:02 2022] POST /identity/v3/domains => generated 267 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:34:04.708755 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53729c4-622e-4fd3-9e26-4137d2bc86d4 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:04.709343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53729c4-622e-4fd3-9e26-4137d2bc86d4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:04.709343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53729c4-622e-4fd3-9e26-4137d2bc86d4 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:04.710667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 588/1170] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:34:04.715762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fd271-bf92-4d17-801f-0a1c4ad4d6f7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:04.716471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fd271-bf92-4d17-801f-0a1c4ad4d6f7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:04.716573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fd271-bf92-4d17-801f-0a1c4ad4d6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:04.723039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:04.735014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-105fd271-bf92-4d17-801f-0a1c4ad4d6f7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:34:04.737472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-105fd271-bf92-4d17-801f-0a1c4ad4d6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:34:04.806514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 583/1171] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:34:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:34:04.829571 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53729c4-622e-4fd3-9e26-4137d2bc86d4 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:04.933557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53729c4-622e-4fd3-9e26-4137d2bc86d4 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:04.959300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:04.967234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73449165-2c31-45cb-8e06-96e6923aa50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:04.967848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73449165-2c31-45cb-8e06-96e6923aa50c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:04.968125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73449165-2c31-45cb-8e06-96e6923aa50c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:04.968393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73449165-2c31-45cb-8e06-96e6923aa50c None admin] PATH_INFO: `/v3/domains/3aa25e149cfd486298d7a9b8ca3e4a3f` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:04.980314 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73449165-2c31-45cb-8e06-96e6923aa50c None admin] RBAC: Authorizing `identity:get_domain(domain_id=3aa25e149cfd486298d7a9b8ca3e4a3f)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:04.983049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73449165-2c31-45cb-8e06-96e6923aa50c None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:04.984983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 589/1172] 10.222.0.79 () {56 vars in 1183 bytes} [Tue Aug 9 16:34:04 2022] GET /identity/v3/domains/3aa25e149cfd486298d7a9b8ca3e4a3f => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:04.991044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fd271-bf92-4d17-801f-0a1c4ad4d6f7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:05.093017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fd271-bf92-4d17-801f-0a1c4ad4d6f7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:05.122997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:05.127698 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f60a9-6321-494e-a942-35a671c5f806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:05.128291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f60a9-6321-494e-a942-35a671c5f806 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:05.128566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f60a9-6321-494e-a942-35a671c5f806 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:05.128829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f60a9-6321-494e-a942-35a671c5f806 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:05.129454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f60a9-6321-494e-a942-35a671c5f806 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:05.130899 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f60a9-6321-494e-a942-35a671c5f806 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:05.228879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 584/1173] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:34:04 2022] POST /identity/v3/projects => generated 361 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:34:07.399254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7907fac-03ec-4f64-b820-1afe0f14785e None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:07.399812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7907fac-03ec-4f64-b820-1afe0f14785e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:07.399812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7907fac-03ec-4f64-b820-1afe0f14785e None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:07.401134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 590/1174] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:34:07.406547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6835cf-0694-4087-9196-972fd2a28413 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:07.407036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6835cf-0694-4087-9196-972fd2a28413 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:07.407146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6835cf-0694-4087-9196-972fd2a28413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:07.424868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6e6835cf-0694-4087-9196-972fd2a28413 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:34:07.426099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6e6835cf-0694-4087-9196-972fd2a28413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:34:07.492998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 585/1175] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:34:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:34:07.518691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7907fac-03ec-4f64-b820-1afe0f14785e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:07.622528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7907fac-03ec-4f64-b820-1afe0f14785e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:07.655658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c246f7-bf40-46b6-95df-a75756bea658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:07.656195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c246f7-bf40-46b6-95df-a75756bea658 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:07.656473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c246f7-bf40-46b6-95df-a75756bea658 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:07.656742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c246f7-bf40-46b6-95df-a75756bea658 None admin] PATH_INFO: `/v3/domains/3aa25e149cfd486298d7a9b8ca3e4a3f` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:07.657851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c246f7-bf40-46b6-95df-a75756bea658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3aa25e149cfd486298d7a9b8ca3e4a3f)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:07.659321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c246f7-bf40-46b6-95df-a75756bea658 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:07.660887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 591/1176] 10.222.0.79 () {56 vars in 1183 bytes} [Tue Aug 9 16:34:07 2022] GET /identity/v3/domains/3aa25e149cfd486298d7a9b8ca3e4a3f => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:07.666737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6835cf-0694-4087-9196-972fd2a28413 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:07.776007 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6835cf-0694-4087-9196-972fd2a28413 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:07.791042 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:07.810915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d8054-be51-4717-b93a-929f0af7acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:07.811510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d8054-be51-4717-b93a-929f0af7acca None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:07.811786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d8054-be51-4717-b93a-929f0af7acca None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:07.812050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d8054-be51-4717-b93a-929f0af7acca None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:07.812686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d8054-be51-4717-b93a-929f0af7acca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:07.814134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d8054-be51-4717-b93a-929f0af7acca None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:07.902523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 586/1177] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:34:07 2022] POST /identity/v3/users => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:34:09.932835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7024cd8-2ea6-4a91-a38c-f721d0683f4a None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:09.933354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7024cd8-2ea6-4a91-a38c-f721d0683f4a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:09.933354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7024cd8-2ea6-4a91-a38c-f721d0683f4a None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:09.934734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 592/1178] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:34:09.940332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c634ae-2e9f-4b62-8817-a537e1cc3f92 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:09.940769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c634ae-2e9f-4b62-8817-a537e1cc3f92 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:09.940872 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c634ae-2e9f-4b62-8817-a537e1cc3f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:09.960179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d1c634ae-2e9f-4b62-8817-a537e1cc3f92 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:34:09.961362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d1c634ae-2e9f-4b62-8817-a537e1cc3f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:34:10.027746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 587/1179] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:34:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:34:10.051804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7024cd8-2ea6-4a91-a38c-f721d0683f4a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:10.113739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:10.158285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7024cd8-2ea6-4a91-a38c-f721d0683f4a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:10.194189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd05c12-9051-4957-95fd-56af5272b211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:10.194746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd05c12-9051-4957-95fd-56af5272b211 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:10.195020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd05c12-9051-4957-95fd-56af5272b211 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:10.195286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd05c12-9051-4957-95fd-56af5272b211 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:10.206012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd05c12-9051-4957-95fd-56af5272b211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-0dd05c12-9051-4957-95fd-56af5272b211 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:34:10.210619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:34:10.211787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:34:10.213039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 593/1180] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:34:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:34:10.218292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c634ae-2e9f-4b62-8817-a537e1cc3f92 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:10.315706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:10.318631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c634ae-2e9f-4b62-8817-a537e1cc3f92 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:10.352834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b33d2e-8194-4d42-b076-b2840bdcfebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:10.353421 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b33d2e-8194-4d42-b076-b2840bdcfebc None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:10.353688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b33d2e-8194-4d42-b076-b2840bdcfebc None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:10.353970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b33d2e-8194-4d42-b076-b2840bdcfebc None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:10.354622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b33d2e-8194-4d42-b076-b2840bdcfebc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:34:10.354915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b33d2e-8194-4d42-b076-b2840bdcfebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:10.356257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b33d2e-8194-4d42-b076-b2840bdcfebc None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:10.365201 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 588/1181] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:34:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:10.371944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd05c12-9051-4957-95fd-56af5272b211 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:10.473796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd05c12-9051-4957-95fd-56af5272b211 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:10.490806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:10.508300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914ca92-b798-4911-875e-e4bd637e7a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:10.508934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914ca92-b798-4911-875e-e4bd637e7a60 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:10.509209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914ca92-b798-4911-875e-e4bd637e7a60 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:10.509476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914ca92-b798-4911-875e-e4bd637e7a60 None admin] PATH_INFO: `/v3/users/00f99b7d20bb48a4b83ff5e07ea19a16` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:10.525442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914ca92-b798-4911-875e-e4bd637e7a60 None admin] RBAC: Authorizing `identity:get_user(user_id=00f99b7d20bb48a4b83ff5e07ea19a16)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:10.526956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914ca92-b798-4911-875e-e4bd637e7a60 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:10.528649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 594/1182] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:34:10 2022] GET /identity/v3/users/00f99b7d20bb48a4b83ff5e07ea19a16 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:10.534502 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b33d2e-8194-4d42-b076-b2840bdcfebc None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:10.639045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b33d2e-8194-4d42-b076-b2840bdcfebc None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:10.672627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd21eee-bc13-4a4e-9eed-428aec65fb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:10.673214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd21eee-bc13-4a4e-9eed-428aec65fb9f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:10.673583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd21eee-bc13-4a4e-9eed-428aec65fb9f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:10.673861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd21eee-bc13-4a4e-9eed-428aec65fb9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:10.704303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:10.743104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd21eee-bc13-4a4e-9eed-428aec65fb9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:10.744448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd21eee-bc13-4a4e-9eed-428aec65fb9f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:10.846862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 589/1183] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:34:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:34:10.852972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914ca92-b798-4911-875e-e4bd637e7a60 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:10.950555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914ca92-b798-4911-875e-e4bd637e7a60 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:10.983802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae3d00-e6eb-4836-a311-45b99b94bf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:10.984407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae3d00-e6eb-4836-a311-45b99b94bf17 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:10.984679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae3d00-e6eb-4836-a311-45b99b94bf17 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:10.984948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae3d00-e6eb-4836-a311-45b99b94bf17 None admin] PATH_INFO: `/v3/projects/6dbf29ce26bc4bddafb163f65d2fc720` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:10.986061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae3d00-e6eb-4836-a311-45b99b94bf17 None admin] RBAC: Authorizing `identity:get_project(project_id=6dbf29ce26bc4bddafb163f65d2fc720)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:10.987476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae3d00-e6eb-4836-a311-45b99b94bf17 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:10.989090 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 595/1184] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:34:10 2022] GET /identity/v3/projects/6dbf29ce26bc4bddafb163f65d2fc720 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:10.998268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd21eee-bc13-4a4e-9eed-428aec65fb9f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:11.062136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:11.101541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd21eee-bc13-4a4e-9eed-428aec65fb9f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:11.135109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d908f6b-10ac-4656-a459-94928690b6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:11.135697 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d908f6b-10ac-4656-a459-94928690b6a0 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:11.135973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d908f6b-10ac-4656-a459-94928690b6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:11.136243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d908f6b-10ac-4656-a459-94928690b6a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:11.136954 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d908f6b-10ac-4656-a459-94928690b6a0 None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=6dbf29ce26bc4bddafb163f65d2fc720, user.id=00f99b7d20bb48a4b83ff5e07ea19a16) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:34:11.137256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d908f6b-10ac-4656-a459-94928690b6a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:11.138693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d908f6b-10ac-4656-a459-94928690b6a0 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:11.143911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 590/1185] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:34:10 2022] GET /identity/v3/role_assignments?user.id=00f99b7d20bb48a4b83ff5e07ea19a16&scope.project.id=6dbf29ce26bc4bddafb163f65d2fc720&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:13.124090 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ed809-68c8-4e8c-8777-9f0bd934891c None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:13.124631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ed809-68c8-4e8c-8777-9f0bd934891c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:13.124631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ed809-68c8-4e8c-8777-9f0bd934891c None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:13.125997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 596/1186] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:34:13.132282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85411da5-443d-466e-b42d-7ebb854757e9 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:13.132746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85411da5-443d-466e-b42d-7ebb854757e9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:13.132896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85411da5-443d-466e-b42d-7ebb854757e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:13.150741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-85411da5-443d-466e-b42d-7ebb854757e9 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:34:13.151924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-85411da5-443d-466e-b42d-7ebb854757e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:34:13.224735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 591/1187] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:34:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:34:13.248499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ed809-68c8-4e8c-8777-9f0bd934891c None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:13.292494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:13.355648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ed809-68c8-4e8c-8777-9f0bd934891c None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:13.389760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6006b3c-791e-4412-bf02-89be7342a703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:13.390321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6006b3c-791e-4412-bf02-89be7342a703 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:13.390595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6006b3c-791e-4412-bf02-89be7342a703 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:13.390860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6006b3c-791e-4412-bf02-89be7342a703 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:13.401884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6006b3c-791e-4412-bf02-89be7342a703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-e6006b3c-791e-4412-bf02-89be7342a703 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:34:13.406520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:34:13.407695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:34:13.408743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 597/1188] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:34:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:34:13.414659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85411da5-443d-466e-b42d-7ebb854757e9 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:13.470446 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:13.516980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85411da5-443d-466e-b42d-7ebb854757e9 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:13.555238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adfe77-7302-403b-9ef8-b3fe375ddde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:13.555835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adfe77-7302-403b-9ef8-b3fe375ddde8 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:13.556110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adfe77-7302-403b-9ef8-b3fe375ddde8 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:13.556377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adfe77-7302-403b-9ef8-b3fe375ddde8 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:13.557028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adfe77-7302-403b-9ef8-b3fe375ddde8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:34:13.557322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adfe77-7302-403b-9ef8-b3fe375ddde8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:13.558698 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adfe77-7302-403b-9ef8-b3fe375ddde8 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:13.566928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 592/1189] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:34:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:34:13.573069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6006b3c-791e-4412-bf02-89be7342a703 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:13.672851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6006b3c-791e-4412-bf02-89be7342a703 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:13.674574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:13.708553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72da759-c579-4015-986a-ac9cac0b0a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:13.709196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72da759-c579-4015-986a-ac9cac0b0a58 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:13.709436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72da759-c579-4015-986a-ac9cac0b0a58 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:13.710233 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72da759-c579-4015-986a-ac9cac0b0a58 None admin] PATH_INFO: `/v3/users/00f99b7d20bb48a4b83ff5e07ea19a16` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:13.711398 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72da759-c579-4015-986a-ac9cac0b0a58 None admin] RBAC: Authorizing `identity:get_user(user_id=00f99b7d20bb48a4b83ff5e07ea19a16)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:13.712816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72da759-c579-4015-986a-ac9cac0b0a58 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:13.714457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 598/1190] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:34:13 2022] GET /identity/v3/users/00f99b7d20bb48a4b83ff5e07ea19a16 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:13.721037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adfe77-7302-403b-9ef8-b3fe375ddde8 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:13.823099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adfe77-7302-403b-9ef8-b3fe375ddde8 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:13.840220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:13.856661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e453c-a5bb-41f9-82c9-703ea196fd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:13.857716 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e453c-a5bb-41f9-82c9-703ea196fd00 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:13.857716 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e453c-a5bb-41f9-82c9-703ea196fd00 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:13.858067 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e453c-a5bb-41f9-82c9-703ea196fd00 None admin] PATH_INFO: `/v3/projects/6dbf29ce26bc4bddafb163f65d2fc720` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:13.859130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e453c-a5bb-41f9-82c9-703ea196fd00 None admin] RBAC: Authorizing `identity:get_project(project_id=6dbf29ce26bc4bddafb163f65d2fc720)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:13.860541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238e453c-a5bb-41f9-82c9-703ea196fd00 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:13.862207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 593/1191] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:34:13 2022] GET /identity/v3/projects/6dbf29ce26bc4bddafb163f65d2fc720 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:13.868142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72da759-c579-4015-986a-ac9cac0b0a58 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:13.968952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72da759-c579-4015-986a-ac9cac0b0a58 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:14.001231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974073a-424a-4aa1-9e27-09df5fb97ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:14.001870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974073a-424a-4aa1-9e27-09df5fb97ea1 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:14.002169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974073a-424a-4aa1-9e27-09df5fb97ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:14.002433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974073a-424a-4aa1-9e27-09df5fb97ea1 None admin] PATH_INFO: `/v3/projects/6dbf29ce26bc4bddafb163f65d2fc720/users/00f99b7d20bb48a4b83ff5e07ea19a16/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:14.004297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974073a-424a-4aa1-9e27-09df5fb97ea1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6dbf29ce26bc4bddafb163f65d2fc720, user_id=00f99b7d20bb48a4b83ff5e07ea19a16, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:14.005714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974073a-424a-4aa1-9e27-09df5fb97ea1 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:14.074665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 599/1192] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:34:13 2022] PUT /identity/v3/projects/6dbf29ce26bc4bddafb163f65d2fc720/users/00f99b7d20bb48a4b83ff5e07ea19a16/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:34:16.061288 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93eb14-409b-4300-8319-f98cb02219ff None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:16.061845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93eb14-409b-4300-8319-f98cb02219ff None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:16.061845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab93eb14-409b-4300-8319-f98cb02219ff None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:16.063238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 594/1193] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:34:16.068621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866aed1-0c8f-424b-9f53-726aab337b0e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:16.069080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866aed1-0c8f-424b-9f53-726aab337b0e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:16.069185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866aed1-0c8f-424b-9f53-726aab337b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:16.087208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8866aed1-0c8f-424b-9f53-726aab337b0e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:34:16.088385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8866aed1-0c8f-424b-9f53-726aab337b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:34:16.111791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:16.154073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 600/1194] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:34:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:34:16.177833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93eb14-409b-4300-8319-f98cb02219ff None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:16.277848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab93eb14-409b-4300-8319-f98cb02219ff None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:16.311094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b66e4-a8af-4918-b010-84a64ffadaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:16.311750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b66e4-a8af-4918-b010-84a64ffadaef None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:16.311975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b66e4-a8af-4918-b010-84a64ffadaef None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:16.312242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b66e4-a8af-4918-b010-84a64ffadaef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:16.322588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b66e4-a8af-4918-b010-84a64ffadaef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-a85b66e4-a8af-4918-b010-84a64ffadaef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:34:16.327179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:34:16.328294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:34:16.329443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 595/1195] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:34:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:34:16.334298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866aed1-0c8f-424b-9f53-726aab337b0e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:16.439081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866aed1-0c8f-424b-9f53-726aab337b0e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:16.472097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d2c2-8b91-4d35-b9f3-c319d807c540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:16.472682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d2c2-8b91-4d35-b9f3-c319d807c540 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:16.472958 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d2c2-8b91-4d35-b9f3-c319d807c540 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:16.473223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5d2c2-8b91-4d35-b9f3-c319d807c540 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:16.473870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5d2c2-8b91-4d35-b9f3-c319d807c540 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:34:16.474197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5d2c2-8b91-4d35-b9f3-c319d807c540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:16.475558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5d2c2-8b91-4d35-b9f3-c319d807c540 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:16.484010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 601/1196] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:34:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:16.489790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b66e4-a8af-4918-b010-84a64ffadaef None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:16.535671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:16.587209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b66e4-a8af-4918-b010-84a64ffadaef None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:16.620669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614bb80-1416-4035-8771-a70316ec7d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:16.621282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614bb80-1416-4035-8771-a70316ec7d83 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:16.621556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614bb80-1416-4035-8771-a70316ec7d83 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:16.621822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614bb80-1416-4035-8771-a70316ec7d83 None admin] PATH_INFO: `/v3/users/00f99b7d20bb48a4b83ff5e07ea19a16` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:16.622948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614bb80-1416-4035-8771-a70316ec7d83 None admin] RBAC: Authorizing `identity:get_user(user_id=00f99b7d20bb48a4b83ff5e07ea19a16)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:16.624367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614bb80-1416-4035-8771-a70316ec7d83 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:16.626004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 596/1197] 10.222.0.79 () {56 vars in 1177 bytes} [Tue Aug 9 16:34:16 2022] GET /identity/v3/users/00f99b7d20bb48a4b83ff5e07ea19a16 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:16.631989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d2c2-8b91-4d35-b9f3-c319d807c540 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:16.657821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:16.730940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5d2c2-8b91-4d35-b9f3-c319d807c540 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:16.763451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aceed6-497f-400b-9e26-393e2380d199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:16.764037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aceed6-497f-400b-9e26-393e2380d199 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:16.764310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aceed6-497f-400b-9e26-393e2380d199 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:16.764575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aceed6-497f-400b-9e26-393e2380d199 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:16.828694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aceed6-497f-400b-9e26-393e2380d199 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:16.830159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aceed6-497f-400b-9e26-393e2380d199 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:16.853288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:16.925523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 602/1198] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:34:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:34:16.931993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614bb80-1416-4035-8771-a70316ec7d83 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:17.026838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614bb80-1416-4035-8771-a70316ec7d83 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:17.041309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:17.061556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ec173-68c0-4023-a5fb-9915802499dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:17.062195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ec173-68c0-4023-a5fb-9915802499dd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:17.062449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ec173-68c0-4023-a5fb-9915802499dd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:17.062712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ec173-68c0-4023-a5fb-9915802499dd None admin] PATH_INFO: `/v3/projects/6dbf29ce26bc4bddafb163f65d2fc720` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:17.063798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ec173-68c0-4023-a5fb-9915802499dd None admin] RBAC: Authorizing `identity:get_project(project_id=6dbf29ce26bc4bddafb163f65d2fc720)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:17.065358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ec173-68c0-4023-a5fb-9915802499dd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:17.066979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 597/1199] 10.222.0.79 () {56 vars in 1186 bytes} [Tue Aug 9 16:34:16 2022] GET /identity/v3/projects/6dbf29ce26bc4bddafb163f65d2fc720 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:17.073223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aceed6-497f-400b-9e26-393e2380d199 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:34:17.169495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aceed6-497f-400b-9e26-393e2380d199 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:34:17.190673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:17.202065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70026576-d3dc-4e92-b265-1d94e73ef3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:34:17.202646 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70026576-d3dc-4e92-b265-1d94e73ef3ad None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:34:17.204886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70026576-d3dc-4e92-b265-1d94e73ef3ad None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:34:17.204886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70026576-d3dc-4e92-b265-1d94e73ef3ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:34:17.204886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70026576-d3dc-4e92-b265-1d94e73ef3ad None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=6dbf29ce26bc4bddafb163f65d2fc720, user.id=00f99b7d20bb48a4b83ff5e07ea19a16) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:34:17.204886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70026576-d3dc-4e92-b265-1d94e73ef3ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:34:17.205616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70026576-d3dc-4e92-b265-1d94e73ef3ad None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:34:17.211751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 603/1200] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:34:17 2022] GET /identity/v3/role_assignments?user.id=00f99b7d20bb48a4b83ff5e07ea19a16&scope.project.id=6dbf29ce26bc4bddafb163f65d2fc720&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:34:27.074197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=9 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:34:27.222729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=7 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:30.501434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a854d-9d6b-4328-b721-dcea5eb42328 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:30.502071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a854d-9d6b-4328-b721-dcea5eb42328 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:30.502071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a854d-9d6b-4328-b721-dcea5eb42328 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:30.503438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 598/1201] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:35:30.508837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06bd18-ed56-4c19-92da-c2f9e50ef686 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:30.514644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06bd18-ed56-4c19-92da-c2f9e50ef686 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:30.514644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06bd18-ed56-4c19-92da-c2f9e50ef686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:30.530428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ad06bd18-ed56-4c19-92da-c2f9e50ef686 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:35:30.531638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ad06bd18-ed56-4c19-92da-c2f9e50ef686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:35:30.596851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 604/1202] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:35:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:35:30.623414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a854d-9d6b-4328-b721-dcea5eb42328 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:30.731558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a854d-9d6b-4328-b721-dcea5eb42328 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:30.766405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f399d-adf3-456e-b37d-833357b8e957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:30.766940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f399d-adf3-456e-b37d-833357b8e957 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:30.767209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f399d-adf3-456e-b37d-833357b8e957 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:30.767472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f399d-adf3-456e-b37d-833357b8e957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:30.782217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f399d-adf3-456e-b37d-833357b8e957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:30.783675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f399d-adf3-456e-b37d-833357b8e957 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-598f399d-adf3-456e-b37d-833357b8e957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:30.798572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:30.804946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:30.806265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:30.806265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:30.806265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:30.806265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 599/1203] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:30.806578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06bd18-ed56-4c19-92da-c2f9e50ef686 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:30.907894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06bd18-ed56-4c19-92da-c2f9e50ef686 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:30.937275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:30.943475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6dc3d-e914-40c6-a8c9-c1933cf37216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:30.944080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6dc3d-e914-40c6-a8c9-c1933cf37216 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:30.944358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6dc3d-e914-40c6-a8c9-c1933cf37216 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:30.944618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6dc3d-e914-40c6-a8c9-c1933cf37216 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:30.945245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6dc3d-e914-40c6-a8c9-c1933cf37216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:30.945540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6dc3d-e914-40c6-a8c9-c1933cf37216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:30.947416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6dc3d-e914-40c6-a8c9-c1933cf37216 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:30.959748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 605/1204] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:30.966740 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f399d-adf3-456e-b37d-833357b8e957 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:31.022325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:31.075855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f399d-adf3-456e-b37d-833357b8e957 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:31.111306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a99521-825b-47bf-9d39-4e7fd72da2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:31.111902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a99521-825b-47bf-9d39-4e7fd72da2e0 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:31.112175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a99521-825b-47bf-9d39-4e7fd72da2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:31.112444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a99521-825b-47bf-9d39-4e7fd72da2e0 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:31.113069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a99521-825b-47bf-9d39-4e7fd72da2e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:31.114536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a99521-825b-47bf-9d39-4e7fd72da2e0 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:31.225144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 600/1205] 10.222.0.79 () {60 vars in 1132 bytes} [Tue Aug 9 16:35:30 2022] POST /identity/v3/users => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:35:33.260193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf8888-35bf-4074-8328-db07aed1059b None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:33.260786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf8888-35bf-4074-8328-db07aed1059b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:33.260786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf8888-35bf-4074-8328-db07aed1059b None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:33.262133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 606/1206] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:35:33.267354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93059bac-af48-4d6a-acfc-c72e8be75a31 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:33.267750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93059bac-af48-4d6a-acfc-c72e8be75a31 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:33.267899 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93059bac-af48-4d6a-acfc-c72e8be75a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:33.286527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-93059bac-af48-4d6a-acfc-c72e8be75a31 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:35:33.291404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-93059bac-af48-4d6a-acfc-c72e8be75a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:35:33.356775 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 601/1207] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:35:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:35:33.381830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf8888-35bf-4074-8328-db07aed1059b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:33.484450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf8888-35bf-4074-8328-db07aed1059b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:33.518458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1464c-d2d7-493f-80d1-4b69347e1682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:33.518998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1464c-d2d7-493f-80d1-4b69347e1682 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:33.519271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1464c-d2d7-493f-80d1-4b69347e1682 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:33.519537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1464c-d2d7-493f-80d1-4b69347e1682 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:33.533309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1464c-d2d7-493f-80d1-4b69347e1682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-bdf1464c-d2d7-493f-80d1-4b69347e1682 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:35:33.537934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:33.539085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:33.540903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 607/1208] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:35:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:35:33.545825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93059bac-af48-4d6a-acfc-c72e8be75a31 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:33.584683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:33.646176 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93059bac-af48-4d6a-acfc-c72e8be75a31 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:33.681287 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960c3f6-5515-444d-bf2b-ca57cc6c4c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:33.683703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960c3f6-5515-444d-bf2b-ca57cc6c4c5a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:33.683703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960c3f6-5515-444d-bf2b-ca57cc6c4c5a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:33.683703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960c3f6-5515-444d-bf2b-ca57cc6c4c5a None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:33.683703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960c3f6-5515-444d-bf2b-ca57cc6c4c5a None admin] RBAC: Adding query filter params (name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:33.683703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960c3f6-5515-444d-bf2b-ca57cc6c4c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:33.684838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960c3f6-5515-444d-bf2b-ca57cc6c4c5a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:33.692655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 602/1209] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:35:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:33.698590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1464c-d2d7-493f-80d1-4b69347e1682 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:33.750811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:33.801030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1464c-d2d7-493f-80d1-4b69347e1682 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:33.836082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38427350-af3e-4839-9f23-add775cd20c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:33.837461 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38427350-af3e-4839-9f23-add775cd20c2 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:33.837738 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38427350-af3e-4839-9f23-add775cd20c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:33.838027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38427350-af3e-4839-9f23-add775cd20c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:33.849138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38427350-af3e-4839-9f23-add775cd20c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:33.850619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38427350-af3e-4839-9f23-add775cd20c2 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-38427350-af3e-4839-9f23-add775cd20c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:33.865994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:33.866965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:33.867934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:33.867934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:33.867934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:33.867934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 608/1210] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:33.873078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960c3f6-5515-444d-bf2b-ca57cc6c4c5a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:33.981610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:33.984642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960c3f6-5515-444d-bf2b-ca57cc6c4c5a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:34.018423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b843ed-8aaf-491d-8d5e-6e775d71326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:34.019021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b843ed-8aaf-491d-8d5e-6e775d71326a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:34.019299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b843ed-8aaf-491d-8d5e-6e775d71326a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:34.019568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b843ed-8aaf-491d-8d5e-6e775d71326a None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:34.020202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b843ed-8aaf-491d-8d5e-6e775d71326a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:34.020501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b843ed-8aaf-491d-8d5e-6e775d71326a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:34.021854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b843ed-8aaf-491d-8d5e-6e775d71326a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:34.033939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 603/1211] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:34.039887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38427350-af3e-4839-9f23-add775cd20c2 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:34.142661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38427350-af3e-4839-9f23-add775cd20c2 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:34.144162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:34.176631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231276cf-1585-4114-8eb4-d94007e4dd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:34.177350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231276cf-1585-4114-8eb4-d94007e4dd4d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:34.177625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231276cf-1585-4114-8eb4-d94007e4dd4d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:34.177903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231276cf-1585-4114-8eb4-d94007e4dd4d None admin] PATH_INFO: `/v3/users/placement` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:34.185632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231276cf-1585-4114-8eb4-d94007e4dd4d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:34.189715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231276cf-1585-4114-8eb4-d94007e4dd4d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-231276cf-1585-4114-8eb4-d94007e4dd4d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:35:34.198758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:34.199929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:34.201339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 609/1212] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:34.205667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b843ed-8aaf-491d-8d5e-6e775d71326a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:34.307010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b843ed-8aaf-491d-8d5e-6e775d71326a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:34.342952 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c6b36-2937-4612-82a0-e47e55218242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:34.343552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c6b36-2937-4612-82a0-e47e55218242 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:34.343824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c6b36-2937-4612-82a0-e47e55218242 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:34.344089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c6b36-2937-4612-82a0-e47e55218242 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:34.344804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c6b36-2937-4612-82a0-e47e55218242 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:34.345099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c6b36-2937-4612-82a0-e47e55218242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:34.346617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c6b36-2937-4612-82a0-e47e55218242 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:34.363987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 604/1213] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Aug 9 16:35:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:34.370340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231276cf-1585-4114-8eb4-d94007e4dd4d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:34.471517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231276cf-1585-4114-8eb4-d94007e4dd4d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:34.505286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a7f75-3376-431b-853b-705497dae8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:34.505883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a7f75-3376-431b-853b-705497dae8a6 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:34.506187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a7f75-3376-431b-853b-705497dae8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:34.506459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a7f75-3376-431b-853b-705497dae8a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:34.536954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:34.576349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a7f75-3376-431b-853b-705497dae8a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:34.577788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a7f75-3376-431b-853b-705497dae8a6 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:34.682552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 610/1214] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:35:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:35:34.688458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c6b36-2937-4612-82a0-e47e55218242 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:34.707764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:34.789244 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c6b36-2937-4612-82a0-e47e55218242 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:34.825630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e596d-0cce-4b23-84e1-e7426d02b728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:34.826278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e596d-0cce-4b23-84e1-e7426d02b728 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:34.826550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e596d-0cce-4b23-84e1-e7426d02b728 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:34.826820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e596d-0cce-4b23-84e1-e7426d02b728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:34.837454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e596d-0cce-4b23-84e1-e7426d02b728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:34.838898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e596d-0cce-4b23-84e1-e7426d02b728 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-342e596d-0cce-4b23-84e1-e7426d02b728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:34.854361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:34.860412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:34.861531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:34.861531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:34.861531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:34.861531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 605/1215] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:34.861823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a7f75-3376-431b-853b-705497dae8a6 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:34.921385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:34.958840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a7f75-3376-431b-853b-705497dae8a6 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:34.992116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc796ab-d265-4431-9b99-f3331d502fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:34.992856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc796ab-d265-4431-9b99-f3331d502fd3 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:34.993593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc796ab-d265-4431-9b99-f3331d502fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:34.993855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc796ab-d265-4431-9b99-f3331d502fd3 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:34.994622 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc796ab-d265-4431-9b99-f3331d502fd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:34.994917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc796ab-d265-4431-9b99-f3331d502fd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:34.996364 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc796ab-d265-4431-9b99-f3331d502fd3 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:35.007620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 611/1216] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:35.013519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e596d-0cce-4b23-84e1-e7426d02b728 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:35.088547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:35.119526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e596d-0cce-4b23-84e1-e7426d02b728 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:35.152948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f482de0-01f1-4a52-afbc-068424c59d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:35.153559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f482de0-01f1-4a52-afbc-068424c59d6a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:35.153829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f482de0-01f1-4a52-afbc-068424c59d6a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:35.154122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f482de0-01f1-4a52-afbc-068424c59d6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:35.160661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f482de0-01f1-4a52-afbc-068424c59d6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:35.162101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f482de0-01f1-4a52-afbc-068424c59d6a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-0f482de0-01f1-4a52-afbc-068424c59d6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:35:35.173396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:35.174605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:35.175832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:35:35.175832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:35:35.175832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:35:35.175832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:35:35.175832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:35:35.175832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:35:35.175832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:35.175832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:35.175832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 606/1217] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:35:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:35.180485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc796ab-d265-4431-9b99-f3331d502fd3 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:35.285120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc796ab-d265-4431-9b99-f3331d502fd3 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:35.308377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:35.319954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a66e47-495d-4ed8-8ca0-14bb234e08a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:35.320557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a66e47-495d-4ed8-8ca0-14bb234e08a5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:35.320834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a66e47-495d-4ed8-8ca0-14bb234e08a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:35.321100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a66e47-495d-4ed8-8ca0-14bb234e08a5 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:35.321804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a66e47-495d-4ed8-8ca0-14bb234e08a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:35.322140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a66e47-495d-4ed8-8ca0-14bb234e08a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:35.323663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a66e47-495d-4ed8-8ca0-14bb234e08a5 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:35.339539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 612/1218] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:35:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:35.344669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f482de0-01f1-4a52-afbc-068424c59d6a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:35.455797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f482de0-01f1-4a52-afbc-068424c59d6a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:35.485312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:35.492459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f778b33-543e-48c5-941c-ad12bf01f0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:35.492650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f778b33-543e-48c5-941c-ad12bf01f0d1 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:35.492906 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f778b33-543e-48c5-941c-ad12bf01f0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:35.493204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f778b33-543e-48c5-941c-ad12bf01f0d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:35.494058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f778b33-543e-48c5-941c-ad12bf01f0d1 None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=d7bc938254514471801c88761525e883) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:35.494434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f778b33-543e-48c5-941c-ad12bf01f0d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:35.495874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f778b33-543e-48c5-941c-ad12bf01f0d1 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:35.501294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 607/1219] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:35:35 2022] GET /identity/v3/role_assignments?user.id=d7bc938254514471801c88761525e883&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:37.535101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e37860-77e3-4c1b-8049-0372b6f6ee5e None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:37.535672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e37860-77e3-4c1b-8049-0372b6f6ee5e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:37.535672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e37860-77e3-4c1b-8049-0372b6f6ee5e None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:37.537054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 613/1220] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:35:37.543114 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb6025-1a1e-4949-9846-bed4ec38f0ab None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:37.545262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb6025-1a1e-4949-9846-bed4ec38f0ab None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:37.545262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb6025-1a1e-4949-9846-bed4ec38f0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:37.562578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-71eb6025-1a1e-4949-9846-bed4ec38f0ab None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:35:37.564252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-71eb6025-1a1e-4949-9846-bed4ec38f0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:35:37.632507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 608/1221] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:35:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:35:37.656091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e37860-77e3-4c1b-8049-0372b6f6ee5e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:37.756704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e37860-77e3-4c1b-8049-0372b6f6ee5e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:37.793035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3f26e-b5ef-4656-930b-84259d7713ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:37.793573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3f26e-b5ef-4656-930b-84259d7713ba None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:37.793844 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3f26e-b5ef-4656-930b-84259d7713ba None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:37.794138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3f26e-b5ef-4656-930b-84259d7713ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:37.804590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3f26e-b5ef-4656-930b-84259d7713ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-48c3f26e-b5ef-4656-930b-84259d7713ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:35:37.809214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:37.810443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:37.811389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 614/1222] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:35:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:37.816654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb6025-1a1e-4949-9846-bed4ec38f0ab None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:37.918843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb6025-1a1e-4949-9846-bed4ec38f0ab None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:37.942117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:37.953318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b377e-8ea8-4523-b6e4-5eccf9e26ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:37.953938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b377e-8ea8-4523-b6e4-5eccf9e26ce5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:37.954243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b377e-8ea8-4523-b6e4-5eccf9e26ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:37.954509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b377e-8ea8-4523-b6e4-5eccf9e26ce5 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:37.955180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b377e-8ea8-4523-b6e4-5eccf9e26ce5 None admin] RBAC: Adding query filter params (name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:37.955478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b377e-8ea8-4523-b6e4-5eccf9e26ce5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:37.956884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b377e-8ea8-4523-b6e4-5eccf9e26ce5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:37.965616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 609/1223] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:35:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:37.972421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3f26e-b5ef-4656-930b-84259d7713ba None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:38.017468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:38.072524 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3f26e-b5ef-4656-930b-84259d7713ba None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:38.105414 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9c9ac-0fd8-44e4-a1f7-21db9027765b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:38.106051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9c9ac-0fd8-44e4-a1f7-21db9027765b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:38.106332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9c9ac-0fd8-44e4-a1f7-21db9027765b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:38.106603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9c9ac-0fd8-44e4-a1f7-21db9027765b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:38.116243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9c9ac-0fd8-44e4-a1f7-21db9027765b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:38.118199 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9c9ac-0fd8-44e4-a1f7-21db9027765b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-aaa9c9ac-0fd8-44e4-a1f7-21db9027765b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:38.134201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:38.154516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:38.155625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:38.155625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:38.155625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:38.155625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 615/1224] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:38.155905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b377e-8ea8-4523-b6e4-5eccf9e26ce5 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:38.241856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b377e-8ea8-4523-b6e4-5eccf9e26ce5 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:38.275701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad8ca4-afce-4563-b7ab-afa449ea5a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:38.276302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad8ca4-afce-4563-b7ab-afa449ea5a47 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:38.276577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad8ca4-afce-4563-b7ab-afa449ea5a47 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:38.276843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad8ca4-afce-4563-b7ab-afa449ea5a47 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:38.277468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad8ca4-afce-4563-b7ab-afa449ea5a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:38.277761 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad8ca4-afce-4563-b7ab-afa449ea5a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:38.279138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad8ca4-afce-4563-b7ab-afa449ea5a47 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:38.290908 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 610/1225] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:38.297258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9c9ac-0fd8-44e4-a1f7-21db9027765b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:38.392033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:38.398768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9c9ac-0fd8-44e4-a1f7-21db9027765b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:38.433378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45744297-03be-4a70-b780-21f032ca81c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:38.434010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45744297-03be-4a70-b780-21f032ca81c4 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:38.434288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45744297-03be-4a70-b780-21f032ca81c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:38.434553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45744297-03be-4a70-b780-21f032ca81c4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:38.442028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45744297-03be-4a70-b780-21f032ca81c4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:38.443395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45744297-03be-4a70-b780-21f032ca81c4 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-45744297-03be-4a70-b780-21f032ca81c4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:35:38.454560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:38.456027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:38.457278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 616/1226] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:38.462127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad8ca4-afce-4563-b7ab-afa449ea5a47 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:38.511447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:38.570677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad8ca4-afce-4563-b7ab-afa449ea5a47 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:38.605606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266e27d-3582-4078-b7b0-8c2ff3762349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:38.606245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266e27d-3582-4078-b7b0-8c2ff3762349 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:38.606518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266e27d-3582-4078-b7b0-8c2ff3762349 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:38.606783 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266e27d-3582-4078-b7b0-8c2ff3762349 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:38.607506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266e27d-3582-4078-b7b0-8c2ff3762349 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:38.607801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266e27d-3582-4078-b7b0-8c2ff3762349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:38.609213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266e27d-3582-4078-b7b0-8c2ff3762349 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:38.627482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 611/1227] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Aug 9 16:35:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:38.633387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45744297-03be-4a70-b780-21f032ca81c4 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:38.739569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45744297-03be-4a70-b780-21f032ca81c4 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:38.772661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfa861-4ba6-49ae-96cd-30b38ad55b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:38.773271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfa861-4ba6-49ae-96cd-30b38ad55b32 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:38.773539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfa861-4ba6-49ae-96cd-30b38ad55b32 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:38.774347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfa861-4ba6-49ae-96cd-30b38ad55b32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:38.784996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfa861-4ba6-49ae-96cd-30b38ad55b32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:38.786463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfa861-4ba6-49ae-96cd-30b38ad55b32 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-2cdfa861-4ba6-49ae-96cd-30b38ad55b32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:38.800888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:38.810171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:38.811216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:38.811216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:38.811216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:38.811216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 617/1228] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:38.811449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266e27d-3582-4078-b7b0-8c2ff3762349 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:38.910468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266e27d-3582-4078-b7b0-8c2ff3762349 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:38.913068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:38.945254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffd3fd-0c1c-42c6-920d-48a07991fdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:38.945847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffd3fd-0c1c-42c6-920d-48a07991fdf1 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:38.946146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffd3fd-0c1c-42c6-920d-48a07991fdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:38.946410 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffd3fd-0c1c-42c6-920d-48a07991fdf1 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:38.947034 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffd3fd-0c1c-42c6-920d-48a07991fdf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:38.947325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffd3fd-0c1c-42c6-920d-48a07991fdf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:38.948668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffd3fd-0c1c-42c6-920d-48a07991fdf1 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:38.960096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 612/1229] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:38.966776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfa861-4ba6-49ae-96cd-30b38ad55b32 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:38.976065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:39.067553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfa861-4ba6-49ae-96cd-30b38ad55b32 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:39.103267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904a82-969d-44ac-90d3-d9847623e502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:39.103911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904a82-969d-44ac-90d3-d9847623e502 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:39.104187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904a82-969d-44ac-90d3-d9847623e502 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:39.104456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904a82-969d-44ac-90d3-d9847623e502 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:39.111123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8904a82-969d-44ac-90d3-d9847623e502 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:39.112566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8904a82-969d-44ac-90d3-d9847623e502 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-b8904a82-969d-44ac-90d3-d9847623e502 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:35:39.123950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:39.128002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:39.129095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:35:39.129095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:35:39.129095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:35:39.129095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:35:39.129095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:35:39.129095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:35:39.129095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:39.129095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:39.129095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 618/1230] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:35:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:39.131103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffd3fd-0c1c-42c6-920d-48a07991fdf1 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:39.234863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffd3fd-0c1c-42c6-920d-48a07991fdf1 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:39.269389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e32512-d7e7-42e1-a5b5-ae6807dd3c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:39.270009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32512-d7e7-42e1-a5b5-ae6807dd3c09 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:39.270286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32512-d7e7-42e1-a5b5-ae6807dd3c09 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:39.270552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e32512-d7e7-42e1-a5b5-ae6807dd3c09 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:39.271228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e32512-d7e7-42e1-a5b5-ae6807dd3c09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:39.271526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e32512-d7e7-42e1-a5b5-ae6807dd3c09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:39.272926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e32512-d7e7-42e1-a5b5-ae6807dd3c09 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:39.285278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 613/1231] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:35:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:39.291751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904a82-969d-44ac-90d3-d9847623e502 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:39.359931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:39.393852 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904a82-969d-44ac-90d3-d9847623e502 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:39.427464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81469ad-e328-4b80-afd0-646a44873882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:39.428111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81469ad-e328-4b80-afd0-646a44873882 None admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:39.428400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81469ad-e328-4b80-afd0-646a44873882 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:39.428661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81469ad-e328-4b80-afd0-646a44873882 None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/d7bc938254514471801c88761525e883/roles/844d711adf9b4c1f8471a01e6031e1e6` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:39.449885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81469ad-e328-4b80-afd0-646a44873882 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=d7bc938254514471801c88761525e883, role_id=844d711adf9b4c1f8471a01e6031e1e6)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:39.451356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81469ad-e328-4b80-afd0-646a44873882 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:39.573756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 619/1232] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:35:39 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/d7bc938254514471801c88761525e883/roles/844d711adf9b4c1f8471a01e6031e1e6 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:35:41.603740 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034217e-f5c4-40ee-83fb-bc22cec7a3f5 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:41.604289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034217e-f5c4-40ee-83fb-bc22cec7a3f5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:41.604289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034217e-f5c4-40ee-83fb-bc22cec7a3f5 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:41.605563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 614/1233] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:35:41.610790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65085f95-771b-4b4d-b2a0-c83315abd10f None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:41.611288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65085f95-771b-4b4d-b2a0-c83315abd10f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:41.611392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65085f95-771b-4b4d-b2a0-c83315abd10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:41.629173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-65085f95-771b-4b4d-b2a0-c83315abd10f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:35:41.630687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-65085f95-771b-4b4d-b2a0-c83315abd10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:35:41.696601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 620/1234] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:35:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 09 16:35:41.719985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034217e-f5c4-40ee-83fb-bc22cec7a3f5 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:41.746303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:41.823902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034217e-f5c4-40ee-83fb-bc22cec7a3f5 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:41.859047 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b14029-6da0-4932-b25d-963a5bf19c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:41.859769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b14029-6da0-4932-b25d-963a5bf19c6d None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:41.860152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b14029-6da0-4932-b25d-963a5bf19c6d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:41.860501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b14029-6da0-4932-b25d-963a5bf19c6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:41.871846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b14029-6da0-4932-b25d-963a5bf19c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-d8b14029-6da0-4932-b25d-963a5bf19c6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:35:41.876563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:41.877715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:41.878636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 615/1235] 10.222.0.79 () {56 vars in 1102 bytes} [Tue Aug 9 16:35:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:41.884722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65085f95-771b-4b4d-b2a0-c83315abd10f None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:41.930859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:41.984123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65085f95-771b-4b4d-b2a0-c83315abd10f None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:42.018315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecba28-e696-4143-b631-bda9b1547105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:42.018913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecba28-e696-4143-b631-bda9b1547105 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:42.019186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecba28-e696-4143-b631-bda9b1547105 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:42.019451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecba28-e696-4143-b631-bda9b1547105 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:42.020108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eecba28-e696-4143-b631-bda9b1547105 None admin] RBAC: Adding query filter params (name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:42.020403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eecba28-e696-4143-b631-bda9b1547105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:42.021777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eecba28-e696-4143-b631-bda9b1547105 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:42.030368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 621/1236] 10.222.0.79 () {56 vars in 1103 bytes} [Tue Aug 9 16:35:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:42.036194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b14029-6da0-4932-b25d-963a5bf19c6d None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:42.122048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:42.138368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b14029-6da0-4932-b25d-963a5bf19c6d None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:42.170000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9b54f-7391-4495-afbf-b6524c923b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:42.170608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9b54f-7391-4495-afbf-b6524c923b12 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:42.170988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9b54f-7391-4495-afbf-b6524c923b12 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:42.171255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9b54f-7391-4495-afbf-b6524c923b12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:42.181929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9b54f-7391-4495-afbf-b6524c923b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:42.183370 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9b54f-7391-4495-afbf-b6524c923b12 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-bed9b54f-7391-4495-afbf-b6524c923b12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:42.197318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:42.198507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:42.199779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:42.199779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:42.199779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:42.199779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 616/1237] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:42.204258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecba28-e696-4143-b631-bda9b1547105 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:42.303106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eecba28-e696-4143-b631-bda9b1547105 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:42.312711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:42.337732 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfcbd9-ea5f-4f47-894f-e5e6718c8066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:42.338346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfcbd9-ea5f-4f47-894f-e5e6718c8066 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:42.338618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfcbd9-ea5f-4f47-894f-e5e6718c8066 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:42.338877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdfcbd9-ea5f-4f47-894f-e5e6718c8066 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:42.339683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfcbd9-ea5f-4f47-894f-e5e6718c8066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:42.339975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfcbd9-ea5f-4f47-894f-e5e6718c8066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:42.341316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdfcbd9-ea5f-4f47-894f-e5e6718c8066 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:42.353299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 622/1238] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:42.359195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9b54f-7391-4495-afbf-b6524c923b12 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:42.465895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9b54f-7391-4495-afbf-b6524c923b12 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:42.501439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd21af-051d-437c-acac-7c8da05f2a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:42.502111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd21af-051d-437c-acac-7c8da05f2a37 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:42.502384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd21af-051d-437c-acac-7c8da05f2a37 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:42.502646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd21af-051d-437c-acac-7c8da05f2a37 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:42.505280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:42.511489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd21af-051d-437c-acac-7c8da05f2a37 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:42.512910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd21af-051d-437c-acac-7c8da05f2a37 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-61bd21af-051d-437c-acac-7c8da05f2a37 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:35:42.523925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:42.525066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:42.526294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 617/1239] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:42.533137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfcbd9-ea5f-4f47-894f-e5e6718c8066 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:42.632383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdfcbd9-ea5f-4f47-894f-e5e6718c8066 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:42.667697 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c3ea3-075e-486b-a87c-8ef553c9a61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:42.668346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c3ea3-075e-486b-a87c-8ef553c9a61b None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:42.668651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c3ea3-075e-486b-a87c-8ef553c9a61b None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:42.668925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c3ea3-075e-486b-a87c-8ef553c9a61b None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:42.669650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c3ea3-075e-486b-a87c-8ef553c9a61b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:42.669972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c3ea3-075e-486b-a87c-8ef553c9a61b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:42.671678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c3ea3-075e-486b-a87c-8ef553c9a61b None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:42.690143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 623/1240] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Aug 9 16:35:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:42.695892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd21af-051d-437c-acac-7c8da05f2a37 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:42.797977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd21af-051d-437c-acac-7c8da05f2a37 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:42.834220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d50151-a9b2-48a8-8c0f-170e9837457d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:42.834816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50151-a9b2-48a8-8c0f-170e9837457d None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:42.835093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50151-a9b2-48a8-8c0f-170e9837457d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:42.835367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d50151-a9b2-48a8-8c0f-170e9837457d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:42.909979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:42.917116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d50151-a9b2-48a8-8c0f-170e9837457d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:42.918588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d50151-a9b2-48a8-8c0f-170e9837457d None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:43.018559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 618/1241] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:35:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:35:43.025376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c3ea3-075e-486b-a87c-8ef553c9a61b None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:43.052441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:43.122061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c3ea3-075e-486b-a87c-8ef553c9a61b None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:43.154262 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ed8a4-5c34-46bc-844e-04a498d88888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:43.154857 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ed8a4-5c34-46bc-844e-04a498d88888 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:43.155128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ed8a4-5c34-46bc-844e-04a498d88888 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:43.155391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ed8a4-5c34-46bc-844e-04a498d88888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:43.165596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ed8a4-5c34-46bc-844e-04a498d88888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:43.174525 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ed8a4-5c34-46bc-844e-04a498d88888 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-d62ed8a4-5c34-46bc-844e-04a498d88888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:43.181083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:43.182458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:43.183503 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:43.183503 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:43.183503 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:43.183503 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 624/1242] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:43.188140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d50151-a9b2-48a8-8c0f-170e9837457d None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:43.283973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:43.289852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d50151-a9b2-48a8-8c0f-170e9837457d None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:43.323245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5701c-bd5c-42c6-b9e7-f8b4f01078e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:43.323755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5701c-bd5c-42c6-b9e7-f8b4f01078e0 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:43.324022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5701c-bd5c-42c6-b9e7-f8b4f01078e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:43.324282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5701c-bd5c-42c6-b9e7-f8b4f01078e0 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:43.324909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5701c-bd5c-42c6-b9e7-f8b4f01078e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:43.325203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5701c-bd5c-42c6-b9e7-f8b4f01078e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:43.326572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5701c-bd5c-42c6-b9e7-f8b4f01078e0 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:43.338488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 619/1243] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:43.344496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ed8a4-5c34-46bc-844e-04a498d88888 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:43.425696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:43.447412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ed8a4-5c34-46bc-844e-04a498d88888 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:43.480581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3f193-cd28-4777-9e7d-f09c05196473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:43.485650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3f193-cd28-4777-9e7d-f09c05196473 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:43.485650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3f193-cd28-4777-9e7d-f09c05196473 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:43.485650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3f193-cd28-4777-9e7d-f09c05196473 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:43.488881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3f193-cd28-4777-9e7d-f09c05196473 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:43.490368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3f193-cd28-4777-9e7d-f09c05196473 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-b9d3f193-cd28-4777-9e7d-f09c05196473 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:35:43.504573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:43.505583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:43.506537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:35:43.506537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:35:43.506537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:35:43.506537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:35:43.506537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:35:43.506537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:35:43.506537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:43.506537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:43.506537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 625/1244] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:35:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:43.512842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5701c-bd5c-42c6-b9e7-f8b4f01078e0 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:43.615104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5701c-bd5c-42c6-b9e7-f8b4f01078e0 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:43.650135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee55ef-ad91-4b87-94d2-7672970cb69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:43.650732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee55ef-ad91-4b87-94d2-7672970cb69d None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:43.650999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee55ef-ad91-4b87-94d2-7672970cb69d None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:43.651261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee55ef-ad91-4b87-94d2-7672970cb69d None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:43.651927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee55ef-ad91-4b87-94d2-7672970cb69d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:43.652219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee55ef-ad91-4b87-94d2-7672970cb69d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:43.653615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee55ef-ad91-4b87-94d2-7672970cb69d None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:43.665843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 620/1245] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:35:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:43.672400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3f193-cd28-4777-9e7d-f09c05196473 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:43.771860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3f193-cd28-4777-9e7d-f09c05196473 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:43.805811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cd897-1779-4eba-b81c-79596bbe6f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:43.806427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cd897-1779-4eba-b81c-79596bbe6f20 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:43.806700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cd897-1779-4eba-b81c-79596bbe6f20 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:43.806963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cd897-1779-4eba-b81c-79596bbe6f20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:43.807671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cd897-1779-4eba-b81c-79596bbe6f20 None admin] RBAC: Adding query filter params (role.id=844d711adf9b4c1f8471a01e6031e1e6, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=d7bc938254514471801c88761525e883) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:43.807968 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cd897-1779-4eba-b81c-79596bbe6f20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:43.809381 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cd897-1779-4eba-b81c-79596bbe6f20 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:43.815004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 626/1246] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:35:43 2022] GET /identity/v3/role_assignments?user.id=d7bc938254514471801c88761525e883&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=844d711adf9b4c1f8471a01e6031e1e6 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:45.962154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee28209-8652-4e35-b1af-a2c4373a9f54 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:45.962726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee28209-8652-4e35-b1af-a2c4373a9f54 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:45.962726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee28209-8652-4e35-b1af-a2c4373a9f54 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:45.963913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 621/1247] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:35:45.969200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb54ce4-35fe-47cd-8c63-3a1a83a5f0d5 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:45.969648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb54ce4-35fe-47cd-8c63-3a1a83a5f0d5 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:45.969743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb54ce4-35fe-47cd-8c63-3a1a83a5f0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:45.976134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:45.988413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-0eb54ce4-35fe-47cd-8c63-3a1a83a5f0d5 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:35:45.993837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0eb54ce4-35fe-47cd-8c63-3a1a83a5f0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:35:46.058296 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 627/1248] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:35:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:35:46.081550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee28209-8652-4e35-b1af-a2c4373a9f54 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:46.100671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:46.185058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee28209-8652-4e35-b1af-a2c4373a9f54 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:46.218460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832f563-2c62-45fb-b3f8-4dccf63e01c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:46.219015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832f563-2c62-45fb-b3f8-4dccf63e01c2 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:46.219290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832f563-2c62-45fb-b3f8-4dccf63e01c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:46.219559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832f563-2c62-45fb-b3f8-4dccf63e01c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:46.229892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832f563-2c62-45fb-b3f8-4dccf63e01c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-7832f563-2c62-45fb-b3f8-4dccf63e01c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:35:46.234484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:46.240243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:46.241326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 622/1249] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:35:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:46.242219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb54ce4-35fe-47cd-8c63-3a1a83a5f0d5 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:46.339923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb54ce4-35fe-47cd-8c63-3a1a83a5f0d5 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:46.367926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:46.372833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec53b96-7d33-48a5-b3b9-2d18933d4994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:46.373469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec53b96-7d33-48a5-b3b9-2d18933d4994 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:46.373743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec53b96-7d33-48a5-b3b9-2d18933d4994 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:46.374037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec53b96-7d33-48a5-b3b9-2d18933d4994 None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:46.374694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec53b96-7d33-48a5-b3b9-2d18933d4994 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:46.375024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec53b96-7d33-48a5-b3b9-2d18933d4994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:46.376446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec53b96-7d33-48a5-b3b9-2d18933d4994 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:46.385534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 628/1250] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:35:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:46.391576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832f563-2c62-45fb-b3f8-4dccf63e01c2 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:46.468878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:46.490989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832f563-2c62-45fb-b3f8-4dccf63e01c2 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:46.525858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe7bf-91ae-40b3-aba5-f8b03d34799a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:46.526500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe7bf-91ae-40b3-aba5-f8b03d34799a None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:46.526780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe7bf-91ae-40b3-aba5-f8b03d34799a None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:46.527052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe7bf-91ae-40b3-aba5-f8b03d34799a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:46.538696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fe7bf-91ae-40b3-aba5-f8b03d34799a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:46.540133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fe7bf-91ae-40b3-aba5-f8b03d34799a None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-026fe7bf-91ae-40b3-aba5-f8b03d34799a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:46.554402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:46.555542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:46.556433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:46.556433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:46.556433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:46.556433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 623/1251] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:46.561533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec53b96-7d33-48a5-b3b9-2d18933d4994 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:46.672917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec53b96-7d33-48a5-b3b9-2d18933d4994 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:46.709504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c60d7-c2d5-4379-b8f3-aa01fb787aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:46.710149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c60d7-c2d5-4379-b8f3-aa01fb787aac None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:46.710446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c60d7-c2d5-4379-b8f3-aa01fb787aac None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:46.714513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c60d7-c2d5-4379-b8f3-aa01fb787aac None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:46.714513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c60d7-c2d5-4379-b8f3-aa01fb787aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:46.714513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c60d7-c2d5-4379-b8f3-aa01fb787aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:46.714513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c60d7-c2d5-4379-b8f3-aa01fb787aac None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:46.725500 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 629/1252] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:35:46.731320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe7bf-91ae-40b3-aba5-f8b03d34799a None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:46.836463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe7bf-91ae-40b3-aba5-f8b03d34799a None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:46.862478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:46.870740 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84518945-f520-431b-807b-79b01424eccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:46.871352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84518945-f520-431b-807b-79b01424eccd None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:46.871627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84518945-f520-431b-807b-79b01424eccd None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:46.871891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84518945-f520-431b-807b-79b01424eccd None admin] PATH_INFO: `/v3/users/placement` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:46.879292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84518945-f520-431b-807b-79b01424eccd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:46.880701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84518945-f520-431b-807b-79b01424eccd None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-84518945-f520-431b-807b-79b01424eccd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:35:46.892193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:46.893468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:46.894696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 624/1253] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:46.899140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c60d7-c2d5-4379-b8f3-aa01fb787aac None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:46.954198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:46.996357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c60d7-c2d5-4379-b8f3-aa01fb787aac None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:47.031121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3eac8-097b-446a-877f-35d3665217ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:47.031708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3eac8-097b-446a-877f-35d3665217ec None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:47.031978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3eac8-097b-446a-877f-35d3665217ec None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:47.032240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3eac8-097b-446a-877f-35d3665217ec None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:47.032983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3eac8-097b-446a-877f-35d3665217ec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:47.033275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3eac8-097b-446a-877f-35d3665217ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:47.034756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3eac8-097b-446a-877f-35d3665217ec None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:47.052700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 630/1254] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Aug 9 16:35:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:47.059223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84518945-f520-431b-807b-79b01424eccd None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:47.161552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84518945-f520-431b-807b-79b01424eccd None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:47.197538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe660e-9e6c-4968-b912-dc949917b42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:47.198165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe660e-9e6c-4968-b912-dc949917b42b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:47.198444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe660e-9e6c-4968-b912-dc949917b42b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:47.198759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe660e-9e6c-4968-b912-dc949917b42b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:47.258647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:47.270828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe660e-9e6c-4968-b912-dc949917b42b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:47.272304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe660e-9e6c-4968-b912-dc949917b42b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:47.379064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 625/1255] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:35:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:35:47.385361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3eac8-097b-446a-877f-35d3665217ec None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:47.504986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3eac8-097b-446a-877f-35d3665217ec None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:47.523974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:47.550230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc48d7-7d29-42f7-af4d-3a605fa04ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:47.551000 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc48d7-7d29-42f7-af4d-3a605fa04ad5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:47.551369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc48d7-7d29-42f7-af4d-3a605fa04ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:47.551727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc48d7-7d29-42f7-af4d-3a605fa04ad5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:47.564197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc48d7-7d29-42f7-af4d-3a605fa04ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:47.566402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc48d7-7d29-42f7-af4d-3a605fa04ad5 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-d2cc48d7-7d29-42f7-af4d-3a605fa04ad5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:47.582754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:47.587326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:47.588287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:47.588287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:47.588287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:47.588287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 631/1256] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:47.590712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe660e-9e6c-4968-b912-dc949917b42b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:47.682092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:47.693847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe660e-9e6c-4968-b912-dc949917b42b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:47.730280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b88ae-6d4c-49d7-adda-dbe189f56d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:47.730928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b88ae-6d4c-49d7-adda-dbe189f56d51 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:47.731196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b88ae-6d4c-49d7-adda-dbe189f56d51 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:47.731463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b88ae-6d4c-49d7-adda-dbe189f56d51 None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:47.732100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b88ae-6d4c-49d7-adda-dbe189f56d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:47.732397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b88ae-6d4c-49d7-adda-dbe189f56d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:47.733754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b88ae-6d4c-49d7-adda-dbe189f56d51 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:47.751169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 626/1257] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:47.757104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc48d7-7d29-42f7-af4d-3a605fa04ad5 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:47.870468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc48d7-7d29-42f7-af4d-3a605fa04ad5 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:47.905130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf704c-a2a1-4ede-8d0d-bfaced83e09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:47.905752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf704c-a2a1-4ede-8d0d-bfaced83e09f None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:47.906179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf704c-a2a1-4ede-8d0d-bfaced83e09f None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:47.906320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf704c-a2a1-4ede-8d0d-bfaced83e09f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:47.912901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf704c-a2a1-4ede-8d0d-bfaced83e09f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:47.914565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf704c-a2a1-4ede-8d0d-bfaced83e09f None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-64cf704c-a2a1-4ede-8d0d-bfaced83e09f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:35:47.926415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:47.930006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:47.931329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:35:47.931329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:35:47.931329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:35:47.931329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:35:47.931329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:35:47.931329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:35:47.931329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:47.931329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:47.931329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 632/1258] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:35:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:47.934054 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b88ae-6d4c-49d7-adda-dbe189f56d51 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:48.051893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b88ae-6d4c-49d7-adda-dbe189f56d51 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:48.090184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc19734-68be-404c-bbe5-12775d1f2ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:48.090856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc19734-68be-404c-bbe5-12775d1f2ad5 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:48.091056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc19734-68be-404c-bbe5-12775d1f2ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:48.091318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc19734-68be-404c-bbe5-12775d1f2ad5 None admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:48.092007 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc19734-68be-404c-bbe5-12775d1f2ad5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:48.092303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc19734-68be-404c-bbe5-12775d1f2ad5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:48.093878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc19734-68be-404c-bbe5-12775d1f2ad5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:48.107177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 627/1259] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:35:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:35:48.113382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf704c-a2a1-4ede-8d0d-bfaced83e09f None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:48.146731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:48.237077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf704c-a2a1-4ede-8d0d-bfaced83e09f None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:48.274763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c934ae0-5aa5-4da6-81b8-593304ad35bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:48.275376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c934ae0-5aa5-4da6-81b8-593304ad35bc None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:48.275655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c934ae0-5aa5-4da6-81b8-593304ad35bc None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:48.275926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c934ae0-5aa5-4da6-81b8-593304ad35bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:48.276680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c934ae0-5aa5-4da6-81b8-593304ad35bc None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=d7bc938254514471801c88761525e883) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:48.276980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c934ae0-5aa5-4da6-81b8-593304ad35bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:48.278449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c934ae0-5aa5-4da6-81b8-593304ad35bc None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:48.284536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 633/1260] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:35:48 2022] GET /identity/v3/role_assignments?user.id=d7bc938254514471801c88761525e883&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:50.361819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38ba8-22b8-4a20-b4c3-960e350f26d3 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:50.362424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38ba8-22b8-4a20-b4c3-960e350f26d3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:50.362424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38ba8-22b8-4a20-b4c3-960e350f26d3 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:50.363810 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 628/1261] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:35:50.370577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a1fe4-1fb2-4f3e-89f4-02a3434f6bb4 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:50.371137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a1fe4-1fb2-4f3e-89f4-02a3434f6bb4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:50.371137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a1fe4-1fb2-4f3e-89f4-02a3434f6bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:50.390300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-bd0a1fe4-1fb2-4f3e-89f4-02a3434f6bb4 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:35:50.400008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bd0a1fe4-1fb2-4f3e-89f4-02a3434f6bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:35:50.461319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 634/1262] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:35:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:35:50.483831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea38ba8-22b8-4a20-b4c3-960e350f26d3 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:50.496711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:50.608800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea38ba8-22b8-4a20-b4c3-960e350f26d3 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:50.647971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199ef9b-4c9a-4236-93d3-ce297235d713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:50.648556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199ef9b-4c9a-4236-93d3-ce297235d713 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:50.648836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199ef9b-4c9a-4236-93d3-ce297235d713 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:50.649103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199ef9b-4c9a-4236-93d3-ce297235d713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:50.661562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199ef9b-4c9a-4236-93d3-ce297235d713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-4199ef9b-4c9a-4236-93d3-ce297235d713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:35:50.667107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:50.669310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:50.670374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 629/1263] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:35:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:50.675264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a1fe4-1fb2-4f3e-89f4-02a3434f6bb4 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:50.696187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:50.793397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a1fe4-1fb2-4f3e-89f4-02a3434f6bb4 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:50.832484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a67d39-27ec-4b1e-9de2-d8b82362bbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:50.833170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a67d39-27ec-4b1e-9de2-d8b82362bbbf None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:50.833449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a67d39-27ec-4b1e-9de2-d8b82362bbbf None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:50.833720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a67d39-27ec-4b1e-9de2-d8b82362bbbf None admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:50.834443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a67d39-27ec-4b1e-9de2-d8b82362bbbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:50.834781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a67d39-27ec-4b1e-9de2-d8b82362bbbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:50.836451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a67d39-27ec-4b1e-9de2-d8b82362bbbf None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:50.845418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 635/1264] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:35:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:50.853508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199ef9b-4c9a-4236-93d3-ce297235d713 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:50.922918 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:50.964360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199ef9b-4c9a-4236-93d3-ce297235d713 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:51.003590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca0969-ea05-4d10-bb30-fb5d54b6277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:51.004219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca0969-ea05-4d10-bb30-fb5d54b6277e None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:51.004497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca0969-ea05-4d10-bb30-fb5d54b6277e None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:51.004761 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca0969-ea05-4d10-bb30-fb5d54b6277e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:51.015882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca0969-ea05-4d10-bb30-fb5d54b6277e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:51.017543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca0969-ea05-4d10-bb30-fb5d54b6277e None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-3bca0969-ea05-4d10-bb30-fb5d54b6277e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:51.034436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:51.035710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:51.036588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:51.036588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:51.036588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:51.036588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 630/1265] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:51.042653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a67d39-27ec-4b1e-9de2-d8b82362bbbf None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:51.136070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:51.155830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a67d39-27ec-4b1e-9de2-d8b82362bbbf None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:51.193844 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec049a8-a54d-40e5-952d-46194263dac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:51.194357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec049a8-a54d-40e5-952d-46194263dac8 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:51.194632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec049a8-a54d-40e5-952d-46194263dac8 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:51.195006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec049a8-a54d-40e5-952d-46194263dac8 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:51.195664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec049a8-a54d-40e5-952d-46194263dac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:51.195962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec049a8-a54d-40e5-952d-46194263dac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:51.197346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec049a8-a54d-40e5-952d-46194263dac8 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:51.209902 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 636/1266] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:51.216564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca0969-ea05-4d10-bb30-fb5d54b6277e None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:51.327507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca0969-ea05-4d10-bb30-fb5d54b6277e None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:51.350919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:51.368029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d53c-b3a5-4c19-8466-e7ff4d3e9d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:51.368748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d53c-b3a5-4c19-8466-e7ff4d3e9d0f None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:51.368933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d53c-b3a5-4c19-8466-e7ff4d3e9d0f None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:51.369198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756d53c-b3a5-4c19-8466-e7ff4d3e9d0f None admin] PATH_INFO: `/v3/users/placement` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:51.378383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756d53c-b3a5-4c19-8466-e7ff4d3e9d0f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:51.379839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756d53c-b3a5-4c19-8466-e7ff4d3e9d0f None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-7756d53c-b3a5-4c19-8466-e7ff4d3e9d0f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:35:51.391938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:51.393241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:51.394380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 631/1267] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:51.399836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec049a8-a54d-40e5-952d-46194263dac8 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:51.504621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec049a8-a54d-40e5-952d-46194263dac8 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:51.542501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce257f6f-7594-4718-8e72-a5ab7eeab08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:51.543119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce257f6f-7594-4718-8e72-a5ab7eeab08a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:51.543390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce257f6f-7594-4718-8e72-a5ab7eeab08a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:51.543653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce257f6f-7594-4718-8e72-a5ab7eeab08a None admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:51.544390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce257f6f-7594-4718-8e72-a5ab7eeab08a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:51.544688 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce257f6f-7594-4718-8e72-a5ab7eeab08a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:51.546161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce257f6f-7594-4718-8e72-a5ab7eeab08a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:51.555938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:51.564466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 637/1268] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Aug 9 16:35:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:51.571615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d53c-b3a5-4c19-8466-e7ff4d3e9d0f None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:51.705794 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756d53c-b3a5-4c19-8466-e7ff4d3e9d0f None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:51.742776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b292ab6-f1b8-423a-998d-1a58e4788da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:51.743082 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b292ab6-f1b8-423a-998d-1a58e4788da7 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:51.743379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b292ab6-f1b8-423a-998d-1a58e4788da7 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:51.743674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b292ab6-f1b8-423a-998d-1a58e4788da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:51.754449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b292ab6-f1b8-423a-998d-1a58e4788da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:51.755970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b292ab6-f1b8-423a-998d-1a58e4788da7 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-4b292ab6-f1b8-423a-998d-1a58e4788da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:51.771562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:51.778453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:51.779423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:51.779423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:51.779423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:51.779423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 632/1269] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:51.779772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce257f6f-7594-4718-8e72-a5ab7eeab08a None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:51.889228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce257f6f-7594-4718-8e72-a5ab7eeab08a None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:51.925652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0dea1-6e7f-4e86-9cd6-0e3514c6ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:51.926276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0dea1-6e7f-4e86-9cd6-0e3514c6ae14 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:51.926550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0dea1-6e7f-4e86-9cd6-0e3514c6ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:51.926813 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0dea1-6e7f-4e86-9cd6-0e3514c6ae14 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:51.927447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0dea1-6e7f-4e86-9cd6-0e3514c6ae14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:51.927741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0dea1-6e7f-4e86-9cd6-0e3514c6ae14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:51.929591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0dea1-6e7f-4e86-9cd6-0e3514c6ae14 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:51.941452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 638/1270] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:35:51.950058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b292ab6-f1b8-423a-998d-1a58e4788da7 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:51.985567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:52.061755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b292ab6-f1b8-423a-998d-1a58e4788da7 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:52.106414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e2bb7-0a36-49c6-a24c-66c81f58acbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:52.107060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e2bb7-0a36-49c6-a24c-66c81f58acbf None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:52.107324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e2bb7-0a36-49c6-a24c-66c81f58acbf None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:52.107587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e2bb7-0a36-49c6-a24c-66c81f58acbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:52.115670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e2bb7-0a36-49c6-a24c-66c81f58acbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:52.117606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e2bb7-0a36-49c6-a24c-66c81f58acbf None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-8c9e2bb7-0a36-49c6-a24c-66c81f58acbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:35:52.130827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:52.138425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:52.139645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:35:52.139645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:35:52.139645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:35:52.139645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:35:52.139645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:35:52.139645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:35:52.139645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:52.139645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:52.139645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 633/1271] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:35:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 09 16:35:52.140245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0dea1-6e7f-4e86-9cd6-0e3514c6ae14 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:52.206858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:52.254556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0dea1-6e7f-4e86-9cd6-0e3514c6ae14 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:52.295932 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc51e0-b204-48fb-a4c3-31d3fd7fb2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:52.296584 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc51e0-b204-48fb-a4c3-31d3fd7fb2f0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:52.296826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc51e0-b204-48fb-a4c3-31d3fd7fb2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:52.297093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc51e0-b204-48fb-a4c3-31d3fd7fb2f0 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:52.297795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc51e0-b204-48fb-a4c3-31d3fd7fb2f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:52.298268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc51e0-b204-48fb-a4c3-31d3fd7fb2f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:52.299734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc51e0-b204-48fb-a4c3-31d3fd7fb2f0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:52.326539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 639/1272] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:35:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:52.326746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e2bb7-0a36-49c6-a24c-66c81f58acbf None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:52.448370 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:52.458284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e2bb7-0a36-49c6-a24c-66c81f58acbf None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:52.504026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b19107-4f0d-48af-b705-3edbb78e6f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:52.504755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b19107-4f0d-48af-b705-3edbb78e6f19 None admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:52.505122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b19107-4f0d-48af-b705-3edbb78e6f19 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:52.505394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b19107-4f0d-48af-b705-3edbb78e6f19 None admin] PATH_INFO: `/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/d7bc938254514471801c88761525e883/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:52.509204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b19107-4f0d-48af-b705-3edbb78e6f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ec40a57f27e42ae81c45450e2297664, user_id=d7bc938254514471801c88761525e883, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:52.509508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b19107-4f0d-48af-b705-3edbb78e6f19 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:52.675366 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 634/1273] 10.222.0.79 () {58 vars in 1439 bytes} [Tue Aug 9 16:35:52 2022] PUT /identity/v3/projects/8ec40a57f27e42ae81c45450e2297664/users/d7bc938254514471801c88761525e883/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:35:54.715887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32925157-5a02-47c2-9888-a7cadaf42183 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:54.716493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32925157-5a02-47c2-9888-a7cadaf42183 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:54.716493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32925157-5a02-47c2-9888-a7cadaf42183 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:54.717737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 640/1274] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:35:54.722944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fac17-5129-4483-bb6f-eca692eb6396 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:54.723322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fac17-5129-4483-bb6f-eca692eb6396 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:54.723515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fac17-5129-4483-bb6f-eca692eb6396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:54.741211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b36fac17-5129-4483-bb6f-eca692eb6396 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:35:54.742382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b36fac17-5129-4483-bb6f-eca692eb6396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:35:54.815522 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 635/1275] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:35:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:35:54.840087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32925157-5a02-47c2-9888-a7cadaf42183 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:54.946905 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32925157-5a02-47c2-9888-a7cadaf42183 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:54.963923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:54.980190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b8660-18e3-446a-9bdc-758a370cc69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:54.980795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b8660-18e3-446a-9bdc-758a370cc69c None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:54.981066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b8660-18e3-446a-9bdc-758a370cc69c None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:54.983516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b8660-18e3-446a-9bdc-758a370cc69c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:54.991776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b8660-18e3-446a-9bdc-758a370cc69c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-070b8660-18e3-446a-9bdc-758a370cc69c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 09 16:35:54.996499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:54.997617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:54.998726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 641/1276] 10.222.0.79 () {56 vars in 1096 bytes} [Tue Aug 9 16:35:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:55.003747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fac17-5129-4483-bb6f-eca692eb6396 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:55.086699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:55.104605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fac17-5129-4483-bb6f-eca692eb6396 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:55.139238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd527f1d-0e3c-4a31-8567-adcd517c8435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:55.139829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd527f1d-0e3c-4a31-8567-adcd517c8435 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:55.140103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd527f1d-0e3c-4a31-8567-adcd517c8435 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:55.140368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd527f1d-0e3c-4a31-8567-adcd517c8435 None admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:55.141084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd527f1d-0e3c-4a31-8567-adcd517c8435 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:55.141361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd527f1d-0e3c-4a31-8567-adcd517c8435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:55.142760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd527f1d-0e3c-4a31-8567-adcd517c8435 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:55.151110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 636/1277] 10.222.0.79 () {56 vars in 1099 bytes} [Tue Aug 9 16:35:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:55.156924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b8660-18e3-446a-9bdc-758a370cc69c None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:55.256240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b8660-18e3-446a-9bdc-758a370cc69c None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:55.289946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10ee5f-d5b5-482a-ac04-0e4083bb35aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:55.290580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ee5f-d5b5-482a-ac04-0e4083bb35aa None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:55.290848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ee5f-d5b5-482a-ac04-0e4083bb35aa None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:55.291106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10ee5f-d5b5-482a-ac04-0e4083bb35aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:55.301172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10ee5f-d5b5-482a-ac04-0e4083bb35aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:55.302665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10ee5f-d5b5-482a-ac04-0e4083bb35aa None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-ab10ee5f-d5b5-482a-ac04-0e4083bb35aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:55.316513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:55.317648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:55.318783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:55.318783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:55.318783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:55.318783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 642/1278] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:55.323605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd527f1d-0e3c-4a31-8567-adcd517c8435 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:55.426715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd527f1d-0e3c-4a31-8567-adcd517c8435 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:55.461560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d1dd0-4528-4e98-8dcc-a6bbb9094f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:55.462186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d1dd0-4528-4e98-8dcc-a6bbb9094f8b None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:55.462459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d1dd0-4528-4e98-8dcc-a6bbb9094f8b None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:55.462721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d1dd0-4528-4e98-8dcc-a6bbb9094f8b None admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:55.463351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d1dd0-4528-4e98-8dcc-a6bbb9094f8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:55.463643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d1dd0-4528-4e98-8dcc-a6bbb9094f8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:55.464985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69d1dd0-4528-4e98-8dcc-a6bbb9094f8b None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:55.477648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 637/1279] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:35:55.483577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10ee5f-d5b5-482a-ac04-0e4083bb35aa None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:55.513797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:55.644209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10ee5f-d5b5-482a-ac04-0e4083bb35aa None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:55.644209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc00a8f-90cb-47ca-baae-b73c31e73588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:55.644209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc00a8f-90cb-47ca-baae-b73c31e73588 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:55.644209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc00a8f-90cb-47ca-baae-b73c31e73588 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc00a8f-90cb-47ca-baae-b73c31e73588 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc00a8f-90cb-47ca-baae-b73c31e73588 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc00a8f-90cb-47ca-baae-b73c31e73588 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-bcc00a8f-90cb-47ca-baae-b73c31e73588 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:55.644673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:55.645510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 09 16:35:55.646753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 09 16:35:55.649586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:55.649586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 643/1280] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:55.649766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d1dd0-4528-4e98-8dcc-a6bbb9094f8b None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:55.649766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:55.741189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69d1dd0-4528-4e98-8dcc-a6bbb9094f8b None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:55.775009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2f1b6-c030-4972-bb24-5f6a14a93913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:55.775601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2f1b6-c030-4972-bb24-5f6a14a93913 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:55.775875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2f1b6-c030-4972-bb24-5f6a14a93913 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:55.776141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2f1b6-c030-4972-bb24-5f6a14a93913 None admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:55.776847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2f1b6-c030-4972-bb24-5f6a14a93913 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:55.777141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2f1b6-c030-4972-bb24-5f6a14a93913 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:55.778586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2f1b6-c030-4972-bb24-5f6a14a93913 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:55.795858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 638/1281] 10.222.0.79 () {56 vars in 1143 bytes} [Tue Aug 9 16:35:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:55.801754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc00a8f-90cb-47ca-baae-b73c31e73588 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:55.890764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:55.900883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc00a8f-90cb-47ca-baae-b73c31e73588 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:55.934168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0afb5-ceaa-4eb5-97a3-a8976d6da730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:55.934753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0afb5-ceaa-4eb5-97a3-a8976d6da730 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:55.935027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0afb5-ceaa-4eb5-97a3-a8976d6da730 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:55.935289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0afb5-ceaa-4eb5-97a3-a8976d6da730 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:56.001069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0afb5-ceaa-4eb5-97a3-a8976d6da730 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:56.002468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0afb5-ceaa-4eb5-97a3-a8976d6da730 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:56.088031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:56.097385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 644/1282] 10.222.0.79 () {58 vars in 1282 bytes} [Tue Aug 9 16:35:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 09 16:35:56.103552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2f1b6-c030-4972-bb24-5f6a14a93913 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:56.165979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:56.204239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2f1b6-c030-4972-bb24-5f6a14a93913 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:56.238015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bffd9-ceaa-42e4-9628-773e01db524c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:56.238625 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bffd9-ceaa-42e4-9628-773e01db524c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:56.238893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bffd9-ceaa-42e4-9628-773e01db524c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:56.239194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bffd9-ceaa-42e4-9628-773e01db524c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:56.249561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bffd9-ceaa-42e4-9628-773e01db524c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:56.251167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bffd9-ceaa-42e4-9628-773e01db524c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-067bffd9-ceaa-42e4-9628-773e01db524c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 09 16:35:56.266104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:56.267208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 09 16:35:56.268696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 09 16:35:56.268696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 09 16:35:56.268696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:56.268696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 639/1283] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:56.273202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0afb5-ceaa-4eb5-97a3-a8976d6da730 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:56.375106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0afb5-ceaa-4eb5-97a3-a8976d6da730 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:56.409211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dad51b-ec2c-49c5-b8d7-ff2a68d67ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:56.409921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dad51b-ec2c-49c5-b8d7-ff2a68d67ed1 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:56.410226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dad51b-ec2c-49c5-b8d7-ff2a68d67ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:56.410497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dad51b-ec2c-49c5-b8d7-ff2a68d67ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:56.411132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dad51b-ec2c-49c5-b8d7-ff2a68d67ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:56.411433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dad51b-ec2c-49c5-b8d7-ff2a68d67ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:56.412796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dad51b-ec2c-49c5-b8d7-ff2a68d67ed1 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:56.424530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 645/1284] 10.222.0.79 () {56 vars in 1109 bytes} [Tue Aug 9 16:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:56.431218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bffd9-ceaa-42e4-9628-773e01db524c None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:56.533577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bffd9-ceaa-42e4-9628-773e01db524c None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:56.547674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:56.566905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be79c89-4c71-4667-ba46-e342af3af981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:56.567520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be79c89-4c71-4667-ba46-e342af3af981 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:56.567795 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be79c89-4c71-4667-ba46-e342af3af981 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:56.568063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be79c89-4c71-4667-ba46-e342af3af981 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:56.574818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be79c89-4c71-4667-ba46-e342af3af981 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:56.576254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be79c89-4c71-4667-ba46-e342af3af981 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-7be79c89-4c71-4667-ba46-e342af3af981 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 09 16:35:56.587941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:56.596143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:56.597048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 16:35:56.597048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 16:35:56.597048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 16:35:56.597048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 16:35:56.597048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 16:35:56.597048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 16:35:56.597048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 09 16:35:56.597048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 16:35:56.597048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 640/1285] 10.222.0.79 () {56 vars in 1111 bytes} [Tue Aug 9 16:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:56.597549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dad51b-ec2c-49c5-b8d7-ff2a68d67ed1 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:56.653510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:56.718379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dad51b-ec2c-49c5-b8d7-ff2a68d67ed1 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:56.756625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa14de-4543-486f-b7e4-78f0b9479dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:56.757233 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa14de-4543-486f-b7e4-78f0b9479dc5 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:56.757508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa14de-4543-486f-b7e4-78f0b9479dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:56.758346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa14de-4543-486f-b7e4-78f0b9479dc5 None admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:56.760480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa14de-4543-486f-b7e4-78f0b9479dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:56.760480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa14de-4543-486f-b7e4-78f0b9479dc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:56.761410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa14de-4543-486f-b7e4-78f0b9479dc5 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:56.775210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 646/1286] 10.222.0.79 () {56 vars in 1148 bytes} [Tue Aug 9 16:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:56.784190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be79c89-4c71-4667-ba46-e342af3af981 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:56.901369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be79c89-4c71-4667-ba46-e342af3af981 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:56.942570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98773846-25d4-410b-a615-be7067d5187c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:56.943173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98773846-25d4-410b-a615-be7067d5187c None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:56.943441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98773846-25d4-410b-a615-be7067d5187c None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:56.943704 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98773846-25d4-410b-a615-be7067d5187c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:56.944420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98773846-25d4-410b-a615-be7067d5187c None admin] RBAC: Adding query filter params (role.id=68125a226d1d4e7c90d58b42cf2c1503, scope.project.id=8ec40a57f27e42ae81c45450e2297664, user.id=d7bc938254514471801c88761525e883) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:56.944718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98773846-25d4-410b-a615-be7067d5187c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:56.946659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98773846-25d4-410b-a615-be7067d5187c None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:56.952626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 641/1287] 10.222.0.79 () {56 vars in 1374 bytes} [Tue Aug 9 16:35:56 2022] GET /identity/v3/role_assignments?user.id=d7bc938254514471801c88761525e883&scope.project.id=8ec40a57f27e42ae81c45450e2297664&role.id=68125a226d1d4e7c90d58b42cf2c1503 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:35:59.015509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbac6ba-87e4-4068-82b1-020bf4c5d0f3 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:59.016357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbac6ba-87e4-4068-82b1-020bf4c5d0f3 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:59.016530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbac6ba-87e4-4068-82b1-020bf4c5d0f3 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:59.017933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 647/1288] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:35:59.025319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfeb3a-4fbf-48fa-b1b6-7d13ac7495af None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:59.025828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfeb3a-4fbf-48fa-b1b6-7d13ac7495af None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:59.026252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfeb3a-4fbf-48fa-b1b6-7d13ac7495af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:59.047283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-bbdfeb3a-4fbf-48fa-b1b6-7d13ac7495af None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:35:59.048741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bbdfeb3a-4fbf-48fa-b1b6-7d13ac7495af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:35:59.081362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:59.122683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 642/1289] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:35:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:35:59.144557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbac6ba-87e4-4068-82b1-020bf4c5d0f3 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:59.256400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbac6ba-87e4-4068-82b1-020bf4c5d0f3 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:59.272426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:35:59.294057 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af40154-6ff6-4cb1-bd59-01c7eb741245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:59.294680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af40154-6ff6-4cb1-bd59-01c7eb741245 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:59.294954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af40154-6ff6-4cb1-bd59-01c7eb741245 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:59.295445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af40154-6ff6-4cb1-bd59-01c7eb741245 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:59.295905 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af40154-6ff6-4cb1-bd59-01c7eb741245 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:59.297444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af40154-6ff6-4cb1-bd59-01c7eb741245 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-3af40154-6ff6-4cb1-bd59-01c7eb741245 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:35:59.307581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:35:59.312374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:35:59.313491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 648/1290] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Aug 9 16:35:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:35:59.314947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfeb3a-4fbf-48fa-b1b6-7d13ac7495af None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:59.431586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfeb3a-4fbf-48fa-b1b6-7d13ac7495af None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:59.466795 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfe00f-b381-47df-a640-2ea573f150ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:59.467388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfe00f-b381-47df-a640-2ea573f150ce None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:59.467659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfe00f-b381-47df-a640-2ea573f150ce None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:59.467988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfe00f-b381-47df-a640-2ea573f150ce None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:59.468624 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfe00f-b381-47df-a640-2ea573f150ce None admin] RBAC: Adding query filter params (name=placement) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:59.468915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfe00f-b381-47df-a640-2ea573f150ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:59.470312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfe00f-b381-47df-a640-2ea573f150ce None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:59.482275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 643/1291] 10.222.0.79 () {56 vars in 1116 bytes} [Tue Aug 9 16:35:59 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:35:59.488944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af40154-6ff6-4cb1-bd59-01c7eb741245 None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:35:59.615820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af40154-6ff6-4cb1-bd59-01c7eb741245 None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:35:59.649131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c33c2-83b4-4a8d-9550-a57f15f2bab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:35:59.649711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c33c2-83b4-4a8d-9550-a57f15f2bab0 None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:35:59.650006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c33c2-83b4-4a8d-9550-a57f15f2bab0 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:35:59.650269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c33c2-83b4-4a8d-9550-a57f15f2bab0 None admin] PATH_INFO: `/v3/services` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:35:59.650883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c33c2-83b4-4a8d-9550-a57f15f2bab0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:35:59.651176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c33c2-83b4-4a8d-9550-a57f15f2bab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:35:59.652517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87c33c2-83b4-4a8d-9550-a57f15f2bab0 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:35:59.657897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 649/1292] 10.222.0.79 () {56 vars in 1116 bytes} [Tue Aug 9 16:35:59 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:36:01.686172 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1323746-ce60-40b3-b8d1-25054c9f59db None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:01.686737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1323746-ce60-40b3-b8d1-25054c9f59db None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:01.686737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1323746-ce60-40b3-b8d1-25054c9f59db None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:01.687999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 644/1293] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:36:01.692994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d287ea-5733-4222-831d-ba357f721b2f None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:01.693437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d287ea-5733-4222-831d-ba357f721b2f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:01.693547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d287ea-5733-4222-831d-ba357f721b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:01.713012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d3d287ea-5733-4222-831d-ba357f721b2f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:36:01.714227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d3d287ea-5733-4222-831d-ba357f721b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:36:01.743516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:36:01.781439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 650/1294] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:36:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:36:01.803160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1323746-ce60-40b3-b8d1-25054c9f59db None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:36:01.834393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:36:01.904321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1323746-ce60-40b3-b8d1-25054c9f59db None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:36:01.937942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185d3e6-ccf9-4eea-ba7f-8cd66ca12ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:36:01.938560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185d3e6-ccf9-4eea-ba7f-8cd66ca12ef5 None admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:01.938839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185d3e6-ccf9-4eea-ba7f-8cd66ca12ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:01.939104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185d3e6-ccf9-4eea-ba7f-8cd66ca12ef5 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:01.939736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185d3e6-ccf9-4eea-ba7f-8cd66ca12ef5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:36:01.941081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185d3e6-ccf9-4eea-ba7f-8cd66ca12ef5 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:36:02.048194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 645/1295] 10.222.0.79 () {60 vars in 1141 bytes} [Tue Aug 9 16:36:01 2022] POST /identity/v3/services => generated 246 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:36:04.086681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80f458-6710-4c9f-8d38-7307476e7635 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:04.087238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80f458-6710-4c9f-8d38-7307476e7635 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:04.087238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80f458-6710-4c9f-8d38-7307476e7635 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:04.088635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 651/1296] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:36:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:36:04.096083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe92e7-c649-419b-a248-85a2c31eff4d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:04.096507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe92e7-c649-419b-a248-85a2c31eff4d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:04.096620 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe92e7-c649-419b-a248-85a2c31eff4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:04.126895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-27fe92e7-c649-419b-a248-85a2c31eff4d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:36:04.128097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-27fe92e7-c649-419b-a248-85a2c31eff4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:36:04.212662 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 646/1297] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:36:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:36:04.236566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80f458-6710-4c9f-8d38-7307476e7635 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:36:04.395041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80f458-6710-4c9f-8d38-7307476e7635 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:36:04.433210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea1619-1b27-4d80-b1c2-0193b75ef78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:36:04.433885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea1619-1b27-4d80-b1c2-0193b75ef78d None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:04.434313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea1619-1b27-4d80-b1c2-0193b75ef78d None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:04.434591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea1619-1b27-4d80-b1c2-0193b75ef78d None admin] PATH_INFO: `/v3/services/placement` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:04.435175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea1619-1b27-4d80-b1c2-0193b75ef78d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:36:04.436540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea1619-1b27-4d80-b1c2-0193b75ef78d None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-f6ea1619-1b27-4d80-b1c2-0193b75ef78d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:36:04.446159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:36:04.448231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:36:04.449277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 652/1298] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Aug 9 16:36:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:36:04.453669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe92e7-c649-419b-a248-85a2c31eff4d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:36:04.474402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:36:04.562083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe92e7-c649-419b-a248-85a2c31eff4d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:36:04.610177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba32f9-0432-471c-ac01-493c24ef7009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:36:04.610775 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba32f9-0432-471c-ac01-493c24ef7009 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:04.611050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba32f9-0432-471c-ac01-493c24ef7009 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:04.611315 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba32f9-0432-471c-ac01-493c24ef7009 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:04.611946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba32f9-0432-471c-ac01-493c24ef7009 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:36:04.612244 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba32f9-0432-471c-ac01-493c24ef7009 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:36:04.613606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba32f9-0432-471c-ac01-493c24ef7009 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:36:04.619692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 647/1299] 10.222.0.79 () {56 vars in 1116 bytes} [Tue Aug 9 16:36:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:36:04.627193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea1619-1b27-4d80-b1c2-0193b75ef78d None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:36:04.672118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:36:04.745691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea1619-1b27-4d80-b1c2-0193b75ef78d None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:36:04.783856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffc030-98b3-40fc-b058-d2e7339c3c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:36:04.784447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffc030-98b3-40fc-b058-d2e7339c3c4a None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:04.784718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffc030-98b3-40fc-b058-d2e7339c3c4a None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:04.784981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffc030-98b3-40fc-b058-d2e7339c3c4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:04.785718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffc030-98b3-40fc-b058-d2e7339c3c4a None admin] RBAC: Adding query filter params (interface=public, service_id=e1b57d91f38540a68e8be15dde5a1d4a, region_id=RegionOne) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:36:04.786114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffc030-98b3-40fc-b058-d2e7339c3c4a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:36:04.787663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffc030-98b3-40fc-b058-d2e7339c3c4a None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:36:04.792989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 653/1300] 10.222.0.79 () {56 vars in 1251 bytes} [Tue Aug 9 16:36:04 2022] GET /identity/v3/endpoints?service_id=e1b57d91f38540a68e8be15dde5a1d4a&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:36:04.798974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba32f9-0432-471c-ac01-493c24ef7009 None admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:36:04.891094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:36:04.909633 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba32f9-0432-471c-ac01-493c24ef7009 None admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:36:04.953705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8207569-da01-4f4d-b593-499559164b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:36:04.954254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207569-da01-4f4d-b593-499559164b01 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:04.954535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207569-da01-4f4d-b593-499559164b01 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:04.954800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207569-da01-4f4d-b593-499559164b01 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:04.955413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8207569-da01-4f4d-b593-499559164b01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:36:04.956769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8207569-da01-4f4d-b593-499559164b01 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:36:04.965169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 648/1301] 10.222.0.79 () {56 vars in 1087 bytes} [Tue Aug 9 16:36:04 2022] GET /identity/v3/services => generated 2191 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:36:07.005151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2a2a2-1c10-42ff-a6f1-b66e7fd756cf None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:07.005680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2a2a2-1c10-42ff-a6f1-b66e7fd756cf None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:07.005680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2a2a2-1c10-42ff-a6f1-b66e7fd756cf None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:07.007236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 654/1302] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:36:07.012395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82d433-8814-4d7a-a3af-1f199c1d08cd None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:07.012826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82d433-8814-4d7a-a3af-1f199c1d08cd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:07.012927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82d433-8814-4d7a-a3af-1f199c1d08cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:07.031975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6d82d433-8814-4d7a-a3af-1f199c1d08cd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:36:07.033149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6d82d433-8814-4d7a-a3af-1f199c1d08cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:36:07.099091 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 649/1303] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:36:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 09 16:36:07.122638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2a2a2-1c10-42ff-a6f1-b66e7fd756cf None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:36:07.220212 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:36:07.231312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2a2a2-1c10-42ff-a6f1-b66e7fd756cf None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:36:07.286225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978b633-5c51-4c2b-97af-298e2becc6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:36:07.287117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978b633-5c51-4c2b-97af-298e2becc6cb None admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:07.287585 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978b633-5c51-4c2b-97af-298e2becc6cb None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:07.288040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978b633-5c51-4c2b-97af-298e2becc6cb None admin] PATH_INFO: `/v3/services/placement` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:07.289469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978b633-5c51-4c2b-97af-298e2becc6cb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:36:07.291583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978b633-5c51-4c2b-97af-298e2becc6cb None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application [None req-1978b633-5c51-4c2b-97af-298e2becc6cb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 16:36:07.307007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 16:36:07.308309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application with Lock( Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self._enter() Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application return self.creator() Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application created_value = creator( Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: ERROR keystone.server.flask.application Aug 09 16:36:07.309302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 655/1304] 10.222.0.79 () {56 vars in 1117 bytes} [Tue Aug 9 16:36:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 09 16:36:07.315236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82d433-8814-4d7a-a3af-1f199c1d08cd None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:36:07.394315 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:36:07.417044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82d433-8814-4d7a-a3af-1f199c1d08cd None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:36:07.458569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31eba3-6cf1-4181-99be-e71179547770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:36:07.459166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31eba3-6cf1-4181-99be-e71179547770 None admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:07.459706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31eba3-6cf1-4181-99be-e71179547770 None admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:07.459972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31eba3-6cf1-4181-99be-e71179547770 None admin] PATH_INFO: `/v3/services` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:07.461150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31eba3-6cf1-4181-99be-e71179547770 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:36:07.461440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31eba3-6cf1-4181-99be-e71179547770 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:36:07.462803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31eba3-6cf1-4181-99be-e71179547770 None admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:36:07.467857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 650/1305] 10.222.0.79 () {56 vars in 1116 bytes} [Tue Aug 9 16:36:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:36:07.473918 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978b633-5c51-4c2b-97af-298e2becc6cb None admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:36:07.573842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978b633-5c51-4c2b-97af-298e2becc6cb None admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:36:07.607647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98fd4b-13f5-4e8d-9c5a-3a6e8eadd395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:36:07.608236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98fd4b-13f5-4e8d-9c5a-3a6e8eadd395 None admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:36:07.608542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98fd4b-13f5-4e8d-9c5a-3a6e8eadd395 None admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:36:07.608810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98fd4b-13f5-4e8d-9c5a-3a6e8eadd395 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:36:07.609465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98fd4b-13f5-4e8d-9c5a-3a6e8eadd395 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:36:07.610832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98fd4b-13f5-4e8d-9c5a-3a6e8eadd395 None admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:36:07.710489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 656/1306] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:36:07 2022] POST /identity/v3/endpoints => generated 331 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:36:17.513548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=30 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:36:17.698413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:23.808855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8bd8d-fae8-40f2-9a46-12b36dac9573 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:23.810495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8bd8d-fae8-40f2-9a46-12b36dac9573 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:23.811184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8bd8d-fae8-40f2-9a46-12b36dac9573 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:23.824424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0403-196a-4f3d-a1b2-c6b62f6888d6 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:23.826361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0403-196a-4f3d-a1b2-c6b62f6888d6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:23.827208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0403-196a-4f3d-a1b2-c6b62f6888d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:23.833121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 651/1307] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:23 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:43:23.977884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-b2fe0403-196a-4f3d-a1b2-c6b62f6888d6 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:23.980702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b2fe0403-196a-4f3d-a1b2-c6b62f6888d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:24.218286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 657/1308] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:24.329440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf92ff-8310-482b-a8f1-09b4c4707184 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:24.329440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf92ff-8310-482b-a8f1-09b4c4707184 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:24.329440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf92ff-8310-482b-a8f1-09b4c4707184 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:24.329440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 652/1309] 10.222.0.79 () {54 vars in 965 bytes} [Tue Aug 9 16:43:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:43:24.335695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609495f-a0f7-4e33-8ca4-6cb11f2c2e3d None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:24.335695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609495f-a0f7-4e33-8ca4-6cb11f2c2e3d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:24.335695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609495f-a0f7-4e33-8ca4-6cb11f2c2e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:24.485708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9609495f-a0f7-4e33-8ca4-6cb11f2c2e3d None None] MFA Rules not processed for user `c2f11dcf2c3c41a0b435a90af74c2e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:24.489938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9609495f-a0f7-4e33-8ca4-6cb11f2c2e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:24.538984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:24.648304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 658/1310] 10.222.0.79 () {58 vars in 1062 bytes} [Tue Aug 9 16:43:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:24.659352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf92ff-8310-482b-a8f1-09b4c4707184 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:24.661745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a9bf92ff-8310-482b-a8f1-09b4c4707184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:24.806535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf92ff-8310-482b-a8f1-09b4c4707184 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:24.921263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e655b3a-fcc6-404d-8a7e-89f88952abdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:24.922312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e655b3a-fcc6-404d-8a7e-89f88952abdf service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:24.922786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e655b3a-fcc6-404d-8a7e-89f88952abdf service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:24.923524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e655b3a-fcc6-404d-8a7e-89f88952abdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:24.925125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0e655b3a-fcc6-404d-8a7e-89f88952abdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:24.990293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:25.055752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e655b3a-fcc6-404d-8a7e-89f88952abdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:25.057255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e655b3a-fcc6-404d-8a7e-89f88952abdf service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:25.081586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 653/1311] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:43:28.036925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df778bb7-79b1-4e24-b511-023ffcbda577 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:28.037509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df778bb7-79b1-4e24-b511-023ffcbda577 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:28.037509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df778bb7-79b1-4e24-b511-023ffcbda577 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:28.038758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 659/1312] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:43:28.044377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b62446-66fd-491f-bd76-16c61b28f36d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:28.044826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b62446-66fd-491f-bd76-16c61b28f36d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:28.044953 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b62446-66fd-491f-bd76-16c61b28f36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:28.129974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-10b62446-66fd-491f-bd76-16c61b28f36d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:28.131263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-10b62446-66fd-491f-bd76-16c61b28f36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:28.233901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 654/1313] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:28.275961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df778bb7-79b1-4e24-b511-023ffcbda577 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:28.278216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-df778bb7-79b1-4e24-b511-023ffcbda577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:28.398521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df778bb7-79b1-4e24-b511-023ffcbda577 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:28.471237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:28.473905 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5d7f3-a257-4fa2-a665-27b0b3eaae03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:28.474569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5d7f3-a257-4fa2-a665-27b0b3eaae03 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:28.474847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5d7f3-a257-4fa2-a665-27b0b3eaae03 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:28.475118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5d7f3-a257-4fa2-a665-27b0b3eaae03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:28.476728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-63a5d7f3-a257-4fa2-a665-27b0b3eaae03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:28.606388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5d7f3-a257-4fa2-a665-27b0b3eaae03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:28.607819 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5d7f3-a257-4fa2-a665-27b0b3eaae03 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:28.627546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 660/1314] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:43:31.314178 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f20ab-55d4-438c-a776-6bedcd7b4f98 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:31.314847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f20ab-55d4-438c-a776-6bedcd7b4f98 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:31.314847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f20ab-55d4-438c-a776-6bedcd7b4f98 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:31.316039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 655/1315] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:43:31.322416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caa301-13eb-46ae-bbb5-3094a46c5e32 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:31.322856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caa301-13eb-46ae-bbb5-3094a46c5e32 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:31.322989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caa301-13eb-46ae-bbb5-3094a46c5e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:31.398909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-26caa301-13eb-46ae-bbb5-3094a46c5e32 None None] MFA Rules not processed for user `2407f024efda4194a4845b435b0ac7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:31.400138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-26caa301-13eb-46ae-bbb5-3094a46c5e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:31.456764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:31.511270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 661/1316] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:31.554474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f20ab-55d4-438c-a776-6bedcd7b4f98 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:31.556012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-1b4f20ab-55d4-438c-a776-6bedcd7b4f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:31.567620 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:31.683376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f20ab-55d4-438c-a776-6bedcd7b4f98 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:31.812839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad03f8-6853-4a09-8618-c338b05150e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:31.813590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad03f8-6853-4a09-8618-c338b05150e2 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:31.814157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad03f8-6853-4a09-8618-c338b05150e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:31.814718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad03f8-6853-4a09-8618-c338b05150e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:31.818686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-7aad03f8-6853-4a09-8618-c338b05150e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:31.924170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:31.980275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad03f8-6853-4a09-8618-c338b05150e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:31.981757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad03f8-6853-4a09-8618-c338b05150e2 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:32.004276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 656/1317] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:43:35.083482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b73230-06ee-4f16-aa5f-c8c23b7871b6 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:35.084067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b73230-06ee-4f16-aa5f-c8c23b7871b6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:35.084067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b73230-06ee-4f16-aa5f-c8c23b7871b6 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:35.086176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 662/1318] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:43:35.092090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69927a-215d-4502-8674-6ce41fd9e7d9 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:35.092862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69927a-215d-4502-8674-6ce41fd9e7d9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:35.092862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69927a-215d-4502-8674-6ce41fd9e7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:35.168479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2a69927a-215d-4502-8674-6ce41fd9e7d9 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:35.169849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2a69927a-215d-4502-8674-6ce41fd9e7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:35.301400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 657/1319] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:35.360256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040c9f0-03ad-44cf-a3e0-d7e903a49eff None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:35.360526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040c9f0-03ad-44cf-a3e0-d7e903a49eff None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:35.360919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040c9f0-03ad-44cf-a3e0-d7e903a49eff None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:35.362469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 663/1320] 10.222.0.79 () {54 vars in 965 bytes} [Tue Aug 9 16:43:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:43:35.371687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd10da-c2cf-44f8-811b-2a756b9e3ee5 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:35.371965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd10da-c2cf-44f8-811b-2a756b9e3ee5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:35.372234 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd10da-c2cf-44f8-811b-2a756b9e3ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:35.395962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:35.462483 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9bfd10da-c2cf-44f8-811b-2a756b9e3ee5 None None] MFA Rules not processed for user `c2f11dcf2c3c41a0b435a90af74c2e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:35.463782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9bfd10da-c2cf-44f8-811b-2a756b9e3ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:35.557594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 658/1321] 10.222.0.79 () {58 vars in 1062 bytes} [Tue Aug 9 16:43:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:35.573383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040c9f0-03ad-44cf-a3e0-d7e903a49eff None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:35.576184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2040c9f0-03ad-44cf-a3e0-d7e903a49eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:35.745538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040c9f0-03ad-44cf-a3e0-d7e903a49eff None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:35.833727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b194eef-e6ca-4c1c-9de6-fbc482fa5579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:35.834439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b194eef-e6ca-4c1c-9de6-fbc482fa5579 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:35.834758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b194eef-e6ca-4c1c-9de6-fbc482fa5579 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:35.835118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b194eef-e6ca-4c1c-9de6-fbc482fa5579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:35.836953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1b194eef-e6ca-4c1c-9de6-fbc482fa5579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:35.846214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:35.982442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b194eef-e6ca-4c1c-9de6-fbc482fa5579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:35.984711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b194eef-e6ca-4c1c-9de6-fbc482fa5579 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:36.009438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 664/1322] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:43:39.738503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85edbd-bcc6-4e73-83ec-763f1a1ecb4d None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:39.739208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85edbd-bcc6-4e73-83ec-763f1a1ecb4d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:39.739208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85edbd-bcc6-4e73-83ec-763f1a1ecb4d None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:39.740523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 659/1323] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:43:39.745600 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d99f2-5b8d-436e-8907-bcb3158018da None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:39.746067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d99f2-5b8d-436e-8907-bcb3158018da None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:39.746179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d99f2-5b8d-436e-8907-bcb3158018da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:39.814011 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-fe5d99f2-5b8d-436e-8907-bcb3158018da None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:39.815134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-fe5d99f2-5b8d-436e-8907-bcb3158018da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:39.875223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:39.917884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 665/1324] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:39.955656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85edbd-bcc6-4e73-83ec-763f1a1ecb4d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:39.957929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3a85edbd-bcc6-4e73-83ec-763f1a1ecb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:40.074180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85edbd-bcc6-4e73-83ec-763f1a1ecb4d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:40.091751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:40.178945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93259b9a-1a9f-49d2-9797-4b931eaff469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:40.179552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93259b9a-1a9f-49d2-9797-4b931eaff469 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:40.179825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93259b9a-1a9f-49d2-9797-4b931eaff469 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:40.180090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93259b9a-1a9f-49d2-9797-4b931eaff469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:40.181812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-93259b9a-1a9f-49d2-9797-4b931eaff469 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:40.303769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93259b9a-1a9f-49d2-9797-4b931eaff469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:40.305194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93259b9a-1a9f-49d2-9797-4b931eaff469 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:40.324010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 660/1325] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:43:42.410132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247723d3-fc13-4ef8-9766-e768108c4776 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:42.410672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247723d3-fc13-4ef8-9766-e768108c4776 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:42.410830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247723d3-fc13-4ef8-9766-e768108c4776 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:42.412302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 666/1326] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:43:42.417769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beab02-71bf-49c1-90fc-26b534988486 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:42.418254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beab02-71bf-49c1-90fc-26b534988486 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:42.418348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59beab02-71bf-49c1-90fc-26b534988486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:42.468431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:42.486892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-59beab02-71bf-49c1-90fc-26b534988486 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:42.488093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-59beab02-71bf-49c1-90fc-26b534988486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:42.591094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 661/1327] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:42.631695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247723d3-fc13-4ef8-9766-e768108c4776 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:42.633220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-247723d3-fc13-4ef8-9766-e768108c4776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:42.738944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247723d3-fc13-4ef8-9766-e768108c4776 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:42.814421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418e95f-d9ea-46a5-93a5-8e27e2137451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:42.815068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418e95f-d9ea-46a5-93a5-8e27e2137451 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:42.815340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418e95f-d9ea-46a5-93a5-8e27e2137451 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:42.815608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418e95f-d9ea-46a5-93a5-8e27e2137451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:42.818098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7418e95f-d9ea-46a5-93a5-8e27e2137451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:42.836343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:42.932081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418e95f-d9ea-46a5-93a5-8e27e2137451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:42.933528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418e95f-d9ea-46a5-93a5-8e27e2137451 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:42.952996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 667/1328] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:43:45.075613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675df0e-4930-469e-b24c-a13c8df5222a None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:45.076163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675df0e-4930-469e-b24c-a13c8df5222a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:45.076163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675df0e-4930-469e-b24c-a13c8df5222a None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:45.077458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 662/1329] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:43:45.083437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fed35-b154-404f-9cc1-315c654689d0 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:45.083923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fed35-b154-404f-9cc1-315c654689d0 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:45.084041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fed35-b154-404f-9cc1-315c654689d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:45.152407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c57fed35-b154-404f-9cc1-315c654689d0 None None] MFA Rules not processed for user `2407f024efda4194a4845b435b0ac7b0`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:45.153676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c57fed35-b154-404f-9cc1-315c654689d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:45.226659 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:45.262498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 668/1330] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:43:45.300259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675df0e-4930-469e-b24c-a13c8df5222a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:45.301771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f675df0e-4930-469e-b24c-a13c8df5222a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:45.413903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675df0e-4930-469e-b24c-a13c8df5222a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:45.443735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:45.496528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6d6a7-e762-4d0e-ac53-706fb30b17c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:45.497300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6d6a7-e762-4d0e-ac53-706fb30b17c2 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:45.497469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6d6a7-e762-4d0e-ac53-706fb30b17c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:45.497676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6d6a7-e762-4d0e-ac53-706fb30b17c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:45.499298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d0b6d6a7-e762-4d0e-ac53-706fb30b17c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:45.622765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6d6a7-e762-4d0e-ac53-706fb30b17c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:45.624288 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6d6a7-e762-4d0e-ac53-706fb30b17c2 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:45.646045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 663/1331] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:43:48.218871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfd3e6-390f-443e-9d07-3ba5470bccea None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:48.219416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfd3e6-390f-443e-9d07-3ba5470bccea None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:48.219416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfd3e6-390f-443e-9d07-3ba5470bccea None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:48.220743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 669/1332] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:43:48.226960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e21ad-a344-4fd4-8a9f-46a11e2dd093 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:48.227509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e21ad-a344-4fd4-8a9f-46a11e2dd093 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:48.227509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e21ad-a344-4fd4-8a9f-46a11e2dd093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:48.295632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:48.298840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d49e21ad-a344-4fd4-8a9f-46a11e2dd093 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:48.300161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d49e21ad-a344-4fd4-8a9f-46a11e2dd093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:48.405323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 664/1333] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:43:48.446121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfd3e6-390f-443e-9d07-3ba5470bccea None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:48.447990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d5dfd3e6-390f-443e-9d07-3ba5470bccea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:48.573769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfd3e6-390f-443e-9d07-3ba5470bccea None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:48.647909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd243b-c467-4b79-bf93-6be2574c6201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:48.648518 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd243b-c467-4b79-bf93-6be2574c6201 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:48.648796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd243b-c467-4b79-bf93-6be2574c6201 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:48.649067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd243b-c467-4b79-bf93-6be2574c6201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:48.650890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-45fd243b-c467-4b79-bf93-6be2574c6201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:48.679194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:48.761432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd243b-c467-4b79-bf93-6be2574c6201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:48.762890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd243b-c467-4b79-bf93-6be2574c6201 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:48.781634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 670/1334] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:43:50.838797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea79f-6bb9-4322-b570-e15b05d698e5 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:50.839336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea79f-6bb9-4322-b570-e15b05d698e5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:50.839336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea79f-6bb9-4322-b570-e15b05d698e5 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:50.840552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 665/1335] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:43:50.845748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49506a56-ff71-4611-a735-8782f75f7c94 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:50.846217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49506a56-ff71-4611-a735-8782f75f7c94 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:50.846321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49506a56-ff71-4611-a735-8782f75f7c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:50.913387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-49506a56-ff71-4611-a735-8782f75f7c94 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:50.914629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-49506a56-ff71-4611-a735-8782f75f7c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:50.990365 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:51.013487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 671/1336] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:51.054269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea79f-6bb9-4322-b570-e15b05d698e5 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:51.055791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-256ea79f-6bb9-4322-b570-e15b05d698e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:51.164563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea79f-6bb9-4322-b570-e15b05d698e5 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:51.194154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:51.240032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5be94-333d-4563-aa8c-afc158fe1991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:51.242384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5be94-333d-4563-aa8c-afc158fe1991 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:51.242665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5be94-333d-4563-aa8c-afc158fe1991 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:51.242937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5be94-333d-4563-aa8c-afc158fe1991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:51.244552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0ab5be94-333d-4563-aa8c-afc158fe1991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:51.353308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5be94-333d-4563-aa8c-afc158fe1991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:51.354766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5be94-333d-4563-aa8c-afc158fe1991 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:51.374808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 666/1337] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:43:54.028353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbea8f-7c45-4893-9aa8-0b012ef25e89 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:54.028944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbea8f-7c45-4893-9aa8-0b012ef25e89 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:54.028944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbea8f-7c45-4893-9aa8-0b012ef25e89 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:54.030388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 672/1338] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:43:54.036063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbc821-4472-4e28-977a-410d709a7e05 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:54.036496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbc821-4472-4e28-977a-410d709a7e05 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:54.036612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbc821-4472-4e28-977a-410d709a7e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:54.101584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-17bbc821-4472-4e28-977a-410d709a7e05 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:54.102732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-17bbc821-4472-4e28-977a-410d709a7e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:54.110703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:54.198874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 667/1339] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:54.236196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbea8f-7c45-4893-9aa8-0b012ef25e89 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:54.238345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e0dbea8f-7c45-4893-9aa8-0b012ef25e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:54.342801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbea8f-7c45-4893-9aa8-0b012ef25e89 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:54.415289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af823cce-f4f3-4087-bd28-86cf82b40b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:54.415798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af823cce-f4f3-4087-bd28-86cf82b40b60 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:54.416075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af823cce-f4f3-4087-bd28-86cf82b40b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:54.416343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af823cce-f4f3-4087-bd28-86cf82b40b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:54.417988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-af823cce-f4f3-4087-bd28-86cf82b40b60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:54.452854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:54.528268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af823cce-f4f3-4087-bd28-86cf82b40b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:54.529708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af823cce-f4f3-4087-bd28-86cf82b40b60 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:54.549013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 673/1340] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:43:58.968176 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a9106-a08e-424b-9834-9d0ce579adac None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:58.968689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a9106-a08e-424b-9834-9d0ce579adac None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:58.968827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a9106-a08e-424b-9834-9d0ce579adac None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:58.970236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 668/1341] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:43:58.976639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab3d38-4de9-4944-ae10-2fc50b5c2ccb None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:58.977120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab3d38-4de9-4944-ae10-2fc50b5c2ccb None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:58.977229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab3d38-4de9-4944-ae10-2fc50b5c2ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:59.045711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-41ab3d38-4de9-4944-ae10-2fc50b5c2ccb None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:43:59.046845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-41ab3d38-4de9-4944-ae10-2fc50b5c2ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:59.134549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:59.147321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 674/1342] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:43:59.192536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a9106-a08e-424b-9834-9d0ce579adac None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:43:59.194273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-dc1a9106-a08e-424b-9834-9d0ce579adac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:59.302532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a9106-a08e-424b-9834-9d0ce579adac None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:43:59.343965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:43:59.382566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7bd89-bc2b-4e14-896f-7fc43bbcbfa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:43:59.383174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7bd89-bc2b-4e14-896f-7fc43bbcbfa0 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:43:59.383451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7bd89-bc2b-4e14-896f-7fc43bbcbfa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:43:59.383778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7bd89-bc2b-4e14-896f-7fc43bbcbfa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:43:59.385375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-37b7bd89-bc2b-4e14-896f-7fc43bbcbfa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:43:59.504577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7bd89-bc2b-4e14-896f-7fc43bbcbfa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:43:59.506069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7bd89-bc2b-4e14-896f-7fc43bbcbfa0 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:43:59.526116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 669/1343] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:02.644748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499b9e9-4572-481d-9cb9-a45f60b21b75 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:02.645278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499b9e9-4572-481d-9cb9-a45f60b21b75 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:02.645278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499b9e9-4572-481d-9cb9-a45f60b21b75 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:02.646685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 675/1344] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:02.652208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab605b-1ad3-46d0-b79e-d29fd540bade None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:02.652678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab605b-1ad3-46d0-b79e-d29fd540bade None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:02.652678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab605b-1ad3-46d0-b79e-d29fd540bade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:02.721755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2cab605b-1ad3-46d0-b79e-d29fd540bade None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:02.723048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2cab605b-1ad3-46d0-b79e-d29fd540bade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:02.742167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:02.824360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 670/1345] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:02.863293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499b9e9-4572-481d-9cb9-a45f60b21b75 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:02.864824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0499b9e9-4572-481d-9cb9-a45f60b21b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:03.009680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499b9e9-4572-481d-9cb9-a45f60b21b75 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:03.098691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2874ec81-b46a-418f-aa6f-ea6a1b479a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:03.099301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874ec81-b46a-418f-aa6f-ea6a1b479a6c service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:03.099575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874ec81-b46a-418f-aa6f-ea6a1b479a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:03.099838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874ec81-b46a-418f-aa6f-ea6a1b479a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:03.101582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2874ec81-b46a-418f-aa6f-ea6a1b479a6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:03.165103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:03.250748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874ec81-b46a-418f-aa6f-ea6a1b479a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:03.253589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2874ec81-b46a-418f-aa6f-ea6a1b479a6c service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:03.278064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 676/1346] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:08.021970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a1a27-bddc-4bb2-bd9f-136b5175d924 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:08.022552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a1a27-bddc-4bb2-bd9f-136b5175d924 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:08.022552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a1a27-bddc-4bb2-bd9f-136b5175d924 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:08.024042 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 671/1347] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:08.029646 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7133605-356e-4c7f-8726-7dc2ecaa0bd7 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:08.030216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7133605-356e-4c7f-8726-7dc2ecaa0bd7 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:08.030323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7133605-356e-4c7f-8726-7dc2ecaa0bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:08.108708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-e7133605-356e-4c7f-8726-7dc2ecaa0bd7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:08.110032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e7133605-356e-4c7f-8726-7dc2ecaa0bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:08.221201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:08.225101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 677/1348] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:08.266024 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a1a27-bddc-4bb2-bd9f-136b5175d924 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:08.267613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bd8a1a27-bddc-4bb2-bd9f-136b5175d924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:08.398208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a1a27-bddc-4bb2-bd9f-136b5175d924 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:08.450326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:08.479932 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271e9e7-87b7-492b-abb9-d08b84f2fe7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:08.480544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271e9e7-87b7-492b-abb9-d08b84f2fe7b service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:08.480818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271e9e7-87b7-492b-abb9-d08b84f2fe7b service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:08.481086 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271e9e7-87b7-492b-abb9-d08b84f2fe7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:08.482884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c271e9e7-87b7-492b-abb9-d08b84f2fe7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:08.603395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271e9e7-87b7-492b-abb9-d08b84f2fe7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:08.604770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271e9e7-87b7-492b-abb9-d08b84f2fe7b service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:08.624599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 672/1349] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:13.713681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99064f-c0be-4e0e-83b2-6120dff0b185 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:13.714257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99064f-c0be-4e0e-83b2-6120dff0b185 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:13.714257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99064f-c0be-4e0e-83b2-6120dff0b185 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:13.716210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 678/1350] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 16:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:13.743090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce065d52-a1bc-48fb-9680-22fd779f8688 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:13.743641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce065d52-a1bc-48fb-9680-22fd779f8688 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:13.743974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce065d52-a1bc-48fb-9680-22fd779f8688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:13.902323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ce065d52-a1bc-48fb-9680-22fd779f8688 None None] MFA Rules not processed for user `d7bc938254514471801c88761525e883`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:13.903929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ce065d52-a1bc-48fb-9680-22fd779f8688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:13.924399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:14.110349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 673/1351] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 16:44:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:19.075168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fe298-e6f5-4d6c-b83a-c7ef42b14598 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:19.076038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fe298-e6f5-4d6c-b83a-c7ef42b14598 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:19.076038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fe298-e6f5-4d6c-b83a-c7ef42b14598 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:19.077006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 679/1352] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 16:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:19.084792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00935b8e-6dd4-488a-a0ae-dedf567c2126 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:19.086402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00935b8e-6dd4-488a-a0ae-dedf567c2126 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:19.087185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00935b8e-6dd4-488a-a0ae-dedf567c2126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:19.225252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-00935b8e-6dd4-488a-a0ae-dedf567c2126 None None] MFA Rules not processed for user `d7bc938254514471801c88761525e883`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:19.226524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-00935b8e-6dd4-488a-a0ae-dedf567c2126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:19.306842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:19.353159 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 674/1353] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 16:44:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:44:20.578655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75ae25-6986-4145-a9db-49b60fb409a1 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:20.579243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75ae25-6986-4145-a9db-49b60fb409a1 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:20.579350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75ae25-6986-4145-a9db-49b60fb409a1 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:20.580650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 680/1354] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 16:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:20.586010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44173604-7fff-45a4-b3cb-8540ecfc6e07 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:20.586455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44173604-7fff-45a4-b3cb-8540ecfc6e07 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:20.586569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44173604-7fff-45a4-b3cb-8540ecfc6e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:20.664878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-44173604-7fff-45a4-b3cb-8540ecfc6e07 None None] MFA Rules not processed for user `d7bc938254514471801c88761525e883`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:20.666390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-44173604-7fff-45a4-b3cb-8540ecfc6e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:20.779505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 675/1355] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 16:44:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:24.159978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04f48-0f48-484d-8daa-70f993dedc03 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:24.160504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04f48-0f48-484d-8daa-70f993dedc03 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:24.160504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04f48-0f48-484d-8daa-70f993dedc03 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:24.161825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 681/1356] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:24.166973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2a535-d28b-4fae-80fd-28b82dbc81fc None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:24.167402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2a535-d28b-4fae-80fd-28b82dbc81fc None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:24.167517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2a535-d28b-4fae-80fd-28b82dbc81fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:24.219686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:24.259534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c5a2a535-d28b-4fae-80fd-28b82dbc81fc None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:24.261050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c5a2a535-d28b-4fae-80fd-28b82dbc81fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:24.375257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 676/1357] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:24.434955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bb681-0894-4920-95a0-c4404d858ff9 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:24.435475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bb681-0894-4920-95a0-c4404d858ff9 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:24.435746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bb681-0894-4920-95a0-c4404d858ff9 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:24.437078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 682/1358] 10.222.0.79 () {54 vars in 962 bytes} [Tue Aug 9 16:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:24.445470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f943e-bfa1-4d19-8da8-8bee54ce5942 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:24.445896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f943e-bfa1-4d19-8da8-8bee54ce5942 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:24.446290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f943e-bfa1-4d19-8da8-8bee54ce5942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:24.545837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0f2f943e-bfa1-4d19-8da8-8bee54ce5942 None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:24.547112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0f2f943e-bfa1-4d19-8da8-8bee54ce5942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:24.607521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:24.683570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 677/1359] 10.222.0.79 () {58 vars in 1059 bytes} [Tue Aug 9 16:44:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:24.693674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bb681-0894-4920-95a0-c4404d858ff9 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:24.698514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-727bb681-0894-4920-95a0-c4404d858ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:24.854948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bb681-0894-4920-95a0-c4404d858ff9 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:24.936938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:24.982580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9130cb-71e8-498b-bb5f-0ea0c096fdbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:24.983178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9130cb-71e8-498b-bb5f-0ea0c096fdbf service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:24.983453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9130cb-71e8-498b-bb5f-0ea0c096fdbf service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:24.983722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9130cb-71e8-498b-bb5f-0ea0c096fdbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:24.985296 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5c9130cb-71e8-498b-bb5f-0ea0c096fdbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:25.090433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9130cb-71e8-498b-bb5f-0ea0c096fdbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:25.092186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9130cb-71e8-498b-bb5f-0ea0c096fdbf service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:25.116431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 683/1360] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:27.405150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8063ce-54d5-433a-bbad-dfe904e0fe20 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:27.405677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8063ce-54d5-433a-bbad-dfe904e0fe20 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:27.405677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8063ce-54d5-433a-bbad-dfe904e0fe20 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:27.412820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb500c-6001-4a51-831e-17b2cd696a26 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:27.413373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb500c-6001-4a51-831e-17b2cd696a26 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:27.413373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eb500c-6001-4a51-831e-17b2cd696a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:27.414129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 678/1361] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:27 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:27.487107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-61eb500c-6001-4a51-831e-17b2cd696a26 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:27.488442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-61eb500c-6001-4a51-831e-17b2cd696a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:27.533045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:27.596120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 684/1362] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:27.645577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8063ce-54d5-433a-bbad-dfe904e0fe20 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:27.647225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8e8063ce-54d5-433a-bbad-dfe904e0fe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:27.905522 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8063ce-54d5-433a-bbad-dfe904e0fe20 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:27.923259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:28.055021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fda86-fd86-4cb3-9d27-7e926e1c71d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:28.055717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fda86-fd86-4cb3-9d27-7e926e1c71d1 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:28.056053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fda86-fd86-4cb3-9d27-7e926e1c71d1 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:28.056379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fda86-fd86-4cb3-9d27-7e926e1c71d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:28.058113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bd4fda86-fd86-4cb3-9d27-7e926e1c71d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:28.169462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fda86-fd86-4cb3-9d27-7e926e1c71d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:28.172099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fda86-fd86-4cb3-9d27-7e926e1c71d1 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:28.198558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 679/1363] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:44:30.941208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977829e-8e73-4c8d-a3fe-674cbe2331df None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:30.941733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977829e-8e73-4c8d-a3fe-674cbe2331df None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:30.941733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977829e-8e73-4c8d-a3fe-674cbe2331df None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:30.944966 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 685/1364] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:30.949058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb249b-62c9-4b2b-8a30-27e0ba172b52 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:30.949782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb249b-62c9-4b2b-8a30-27e0ba172b52 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:30.949782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb249b-62c9-4b2b-8a30-27e0ba172b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:30.974050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:31.042817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-72cb249b-62c9-4b2b-8a30-27e0ba172b52 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:31.044210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-72cb249b-62c9-4b2b-8a30-27e0ba172b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:31.185119 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 680/1365] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:44:31.259797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7cc36-6dc8-46b0-8eb0-776b2d0fa070 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:31.260441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7cc36-6dc8-46b0-8eb0-776b2d0fa070 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:31.260815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7cc36-6dc8-46b0-8eb0-776b2d0fa070 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:31.262407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 686/1366] 10.222.0.79 () {54 vars in 962 bytes} [Tue Aug 9 16:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:31.269701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d6f63-7d94-4750-916b-5238f5e6d09e None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:31.270094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d6f63-7d94-4750-916b-5238f5e6d09e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:31.270390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d6f63-7d94-4750-916b-5238f5e6d09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:31.410025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-3b6d6f63-7d94-4750-916b-5238f5e6d09e None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:31.411135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3b6d6f63-7d94-4750-916b-5238f5e6d09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:31.468747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:31.611227 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 681/1367] 10.222.0.79 () {58 vars in 1059 bytes} [Tue Aug 9 16:44:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:31.622657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7cc36-6dc8-46b0-8eb0-776b2d0fa070 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:31.625924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-20c7cc36-6dc8-46b0-8eb0-776b2d0fa070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:31.816768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7cc36-6dc8-46b0-8eb0-776b2d0fa070 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:31.849561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:31.995548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac6d6f-01c7-4267-83fa-021b4fbfaa27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:31.996334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac6d6f-01c7-4267-83fa-021b4fbfaa27 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:31.996709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac6d6f-01c7-4267-83fa-021b4fbfaa27 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:31.997078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac6d6f-01c7-4267-83fa-021b4fbfaa27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:31.998868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a6ac6d6f-01c7-4267-83fa-021b4fbfaa27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:32.112889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac6d6f-01c7-4267-83fa-021b4fbfaa27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:32.114458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ac6d6f-01c7-4267-83fa-021b4fbfaa27 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:32.133984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 687/1368] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:33.007010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa557d-41c3-4528-bf42-a564d29a8682 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:33.007579 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa557d-41c3-4528-bf42-a564d29a8682 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:33.007579 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34aa557d-41c3-4528-bf42-a564d29a8682 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:33.009427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 682/1369] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:33.015251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c59df-ee47-4ce1-9f06-885cfa62054d None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:33.015791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c59df-ee47-4ce1-9f06-885cfa62054d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:33.015955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1c59df-ee47-4ce1-9f06-885cfa62054d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:33.054614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:33.102218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9d1c59df-ee47-4ce1-9f06-885cfa62054d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:33.103556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9d1c59df-ee47-4ce1-9f06-885cfa62054d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:33.250698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 688/1370] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:33.590964 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dae591-53c3-4610-8529-05cf37993a7d None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:33.591274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dae591-53c3-4610-8529-05cf37993a7d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:33.591578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dae591-53c3-4610-8529-05cf37993a7d None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:33.593056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 683/1371] 10.222.0.79 () {54 vars in 964 bytes} [Tue Aug 9 16:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:33.598965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f47a10-61b0-447b-afcc-f86ad518067a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:33.599379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f47a10-61b0-447b-afcc-f86ad518067a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:33.599715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f47a10-61b0-447b-afcc-f86ad518067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:33.694082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a0f47a10-61b0-447b-afcc-f86ad518067a None None] MFA Rules not processed for user `289e7caceab944e9b90f2e9298fb2cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:33.695318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a0f47a10-61b0-447b-afcc-f86ad518067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:33.760084 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:33.806132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 689/1372] 10.222.0.79 () {58 vars in 1061 bytes} [Tue Aug 9 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:33.816970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dae591-53c3-4610-8529-05cf37993a7d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:33.817423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-63dae591-53c3-4610-8529-05cf37993a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:33.958291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dae591-53c3-4610-8529-05cf37993a7d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:34.016417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:34.056440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef3c4c-87b3-42e9-b29e-59263b98bfed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:34.056972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef3c4c-87b3-42e9-b29e-59263b98bfed service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:34.057245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef3c4c-87b3-42e9-b29e-59263b98bfed service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:34.057511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef3c4c-87b3-42e9-b29e-59263b98bfed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:34.059316 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-84ef3c4c-87b3-42e9-b29e-59263b98bfed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:34.181376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef3c4c-87b3-42e9-b29e-59263b98bfed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:34.182850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef3c4c-87b3-42e9-b29e-59263b98bfed service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:34.205765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 684/1373] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:44:34.600689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42591726-32e1-4d10-ac8e-a597b0846533 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:34.603449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42591726-32e1-4d10-ac8e-a597b0846533 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:34.603449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42591726-32e1-4d10-ac8e-a597b0846533 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:34.606962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 690/1374] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:34.623410 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fb5e6-25e2-48b9-9363-ccea3a7f16fd None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:34.623916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fb5e6-25e2-48b9-9363-ccea3a7f16fd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:34.624480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fb5e6-25e2-48b9-9363-ccea3a7f16fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:34.731720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c72fb5e6-25e2-48b9-9363-ccea3a7f16fd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:34.733526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c72fb5e6-25e2-48b9-9363-ccea3a7f16fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:34.773343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:34.870268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 685/1375] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:34.908666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42591726-32e1-4d10-ac8e-a597b0846533 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:34.910294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-42591726-32e1-4d10-ac8e-a597b0846533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:35.131286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42591726-32e1-4d10-ac8e-a597b0846533 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:35.198302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:35.267843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e459fb9-8f81-4308-913d-3409d5cda2e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:35.268488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e459fb9-8f81-4308-913d-3409d5cda2e4 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:35.268773 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e459fb9-8f81-4308-913d-3409d5cda2e4 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:35.269072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e459fb9-8f81-4308-913d-3409d5cda2e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:35.271202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1e459fb9-8f81-4308-913d-3409d5cda2e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:35.384711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e459fb9-8f81-4308-913d-3409d5cda2e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:35.386934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e459fb9-8f81-4308-913d-3409d5cda2e4 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:35.415205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 691/1376] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:38.010040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aae53-4221-49a6-8c23-cca398efff0f None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:38.010956 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aae53-4221-49a6-8c23-cca398efff0f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:38.011230 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498aae53-4221-49a6-8c23-cca398efff0f None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:38.012711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 686/1377] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:38.019209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8474ae-2686-487a-90c6-c3c0c9f08b22 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:38.019980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8474ae-2686-487a-90c6-c3c0c9f08b22 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:38.019980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8474ae-2686-487a-90c6-c3c0c9f08b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:38.097078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8d8474ae-2686-487a-90c6-c3c0c9f08b22 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:38.098368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8d8474ae-2686-487a-90c6-c3c0c9f08b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:38.108116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:38.242523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 692/1378] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:44:38.289243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498aae53-4221-49a6-8c23-cca398efff0f None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:38.290844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-498aae53-4221-49a6-8c23-cca398efff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:38.572240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498aae53-4221-49a6-8c23-cca398efff0f None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:38.577366 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:38.728186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d14a26-2fd7-496c-b888-96fcf80597f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:38.729099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d14a26-2fd7-496c-b888-96fcf80597f5 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:38.729711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d14a26-2fd7-496c-b888-96fcf80597f5 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:38.730210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d14a26-2fd7-496c-b888-96fcf80597f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:38.732988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f5d14a26-2fd7-496c-b888-96fcf80597f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:38.895352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d14a26-2fd7-496c-b888-96fcf80597f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:38.896865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d14a26-2fd7-496c-b888-96fcf80597f5 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:38.926024 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 687/1379] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:44:41.440504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b84f7-49f8-4244-9dd9-697b1e1e3b46 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:41.441047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b84f7-49f8-4244-9dd9-697b1e1e3b46 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:41.441047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b84f7-49f8-4244-9dd9-697b1e1e3b46 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:41.442772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 693/1380] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:41.449399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2f734-a91e-4b3c-b8a4-613a55407ace None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:41.451012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2f734-a91e-4b3c-b8a4-613a55407ace None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:41.452006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2f734-a91e-4b3c-b8a4-613a55407ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:41.472712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:41.555889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9bf2f734-a91e-4b3c-b8a4-613a55407ace None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:41.557261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9bf2f734-a91e-4b3c-b8a4-613a55407ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:41.717556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 688/1381] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:41.765224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b84f7-49f8-4244-9dd9-697b1e1e3b46 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:41.766797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-669b84f7-49f8-4244-9dd9-697b1e1e3b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:41.959064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:41.974032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b84f7-49f8-4244-9dd9-697b1e1e3b46 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:42.101269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc209852-1bea-4fb4-8493-38a12cb05a80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:42.101869 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc209852-1bea-4fb4-8493-38a12cb05a80 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:42.102175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc209852-1bea-4fb4-8493-38a12cb05a80 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:42.102575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc209852-1bea-4fb4-8493-38a12cb05a80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:42.104172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-dc209852-1bea-4fb4-8493-38a12cb05a80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:42.240449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc209852-1bea-4fb4-8493-38a12cb05a80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:42.241941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc209852-1bea-4fb4-8493-38a12cb05a80 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:42.258201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:42.270340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 694/1382] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:44.909646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfb8e7-36b7-4816-bf91-2b334533289d None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:44.910225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfb8e7-36b7-4816-bf91-2b334533289d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:44.910225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfb8e7-36b7-4816-bf91-2b334533289d None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:44.911603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 689/1383] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:44:44.917227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502481e-75d9-4db3-9204-f84900145e26 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:44.917840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502481e-75d9-4db3-9204-f84900145e26 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:44.918067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502481e-75d9-4db3-9204-f84900145e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:44.990701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9502481e-75d9-4db3-9204-f84900145e26 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:44.991906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9502481e-75d9-4db3-9204-f84900145e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:45.102021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 695/1384] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:45.156228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfb8e7-36b7-4816-bf91-2b334533289d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:45.158805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3dbfb8e7-36b7-4816-bf91-2b334533289d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:45.285045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:45.394575 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfb8e7-36b7-4816-bf91-2b334533289d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:45.531653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588b3fe-1a1c-4815-ab56-e23634e464f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:45.533165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588b3fe-1a1c-4815-ab56-e23634e464f7 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:45.533440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588b3fe-1a1c-4815-ab56-e23634e464f7 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:45.533844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588b3fe-1a1c-4815-ab56-e23634e464f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:45.536231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3588b3fe-1a1c-4815-ab56-e23634e464f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:45.594422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:45.637966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588b3fe-1a1c-4815-ab56-e23634e464f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:45.640182 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588b3fe-1a1c-4815-ab56-e23634e464f7 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:45.660475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 690/1385] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:44:47.842681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d362b-eac5-4f17-af50-45ca0d6bef98 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:47.843231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d362b-eac5-4f17-af50-45ca0d6bef98 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:47.843231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d362b-eac5-4f17-af50-45ca0d6bef98 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:47.844551 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 696/1386] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:47.851452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f2876-dcb6-4662-9e1f-74a22fc117a6 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:47.851878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f2876-dcb6-4662-9e1f-74a22fc117a6 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:47.851977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f2876-dcb6-4662-9e1f-74a22fc117a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:47.923479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-dc0f2876-dcb6-4662-9e1f-74a22fc117a6 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:47.924701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-dc0f2876-dcb6-4662-9e1f-74a22fc117a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:48.034431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 691/1387] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:48.080921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d362b-eac5-4f17-af50-45ca0d6bef98 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:48.082927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-481d362b-eac5-4f17-af50-45ca0d6bef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:48.169433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:48.254305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d362b-eac5-4f17-af50-45ca0d6bef98 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:48.392020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282413de-71d0-4834-81e0-6a954abb6ad5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:48.392701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282413de-71d0-4834-81e0-6a954abb6ad5 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:48.392701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282413de-71d0-4834-81e0-6a954abb6ad5 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:48.392864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282413de-71d0-4834-81e0-6a954abb6ad5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:48.394531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-282413de-71d0-4834-81e0-6a954abb6ad5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:48.437696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:48.500305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282413de-71d0-4834-81e0-6a954abb6ad5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:48.501987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282413de-71d0-4834-81e0-6a954abb6ad5 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:48.526856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 697/1388] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:50.857746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dd386-5884-4713-8b12-b335c9e3bd73 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:50.858354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dd386-5884-4713-8b12-b335c9e3bd73 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:50.858354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141dd386-5884-4713-8b12-b335c9e3bd73 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:50.859710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 692/1389] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:50.865277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e66c4f-b355-4ebf-a022-4075e699b3ff None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:50.865742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e66c4f-b355-4ebf-a022-4075e699b3ff None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:50.865843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e66c4f-b355-4ebf-a022-4075e699b3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:50.936081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-b3e66c4f-b355-4ebf-a022-4075e699b3ff None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:50.937274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b3e66c4f-b355-4ebf-a022-4075e699b3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:51.046705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 698/1390] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:51.096337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dd386-5884-4713-8b12-b335c9e3bd73 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:51.097832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-141dd386-5884-4713-8b12-b335c9e3bd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:51.128679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c49b0-dcf6-4323-a440-ed9a8986857c None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:51.128679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c49b0-dcf6-4323-a440-ed9a8986857c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:51.128679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c49b0-dcf6-4323-a440-ed9a8986857c None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:51.129258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 699/1391] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:44:51.133900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:51.135276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417836a1-cd32-47cb-9017-1569f96209e1 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:51.135741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417836a1-cd32-47cb-9017-1569f96209e1 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:51.136017 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417836a1-cd32-47cb-9017-1569f96209e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:51.153103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:51.205774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-417836a1-cd32-47cb-9017-1569f96209e1 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:51.207103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-417836a1-cd32-47cb-9017-1569f96209e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:51.257215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141dd386-5884-4713-8b12-b335c9e3bd73 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:51.318658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 700/1392] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:51.370401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:51.377740 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834794-4c11-41a0-89c2-e4afab3d5d3f None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:51.386582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834794-4c11-41a0-89c2-e4afab3d5d3f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:51.386582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2834794-4c11-41a0-89c2-e4afab3d5d3f None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:51.386582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 701/1393] 10.222.0.79 () {54 vars in 967 bytes} [Tue Aug 9 16:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:44:51.386896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d075b-c631-41c4-aeab-db8850e75fa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:51.386896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d075b-c631-41c4-aeab-db8850e75fa2 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:51.386896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d075b-c631-41c4-aeab-db8850e75fa2 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:51.386896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d075b-c631-41c4-aeab-db8850e75fa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:51.387278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe344f6e-59ad-4273-a131-f73333ae3dd4 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:51.387278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe344f6e-59ad-4273-a131-f73333ae3dd4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:51.387465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-820d075b-c631-41c4-aeab-db8850e75fa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:51.387564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe344f6e-59ad-4273-a131-f73333ae3dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:51.488546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-fe344f6e-59ad-4273-a131-f73333ae3dd4 None None] MFA Rules not processed for user `33003a9bf4454f5bac2e6ad73544efc4`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:51.489764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-fe344f6e-59ad-4273-a131-f73333ae3dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:51.495011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d075b-c631-41c4-aeab-db8850e75fa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:51.497469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d075b-c631-41c4-aeab-db8850e75fa2 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:51.520431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 693/1394] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:51.525097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:51.584445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 702/1395] 10.222.0.79 () {58 vars in 1064 bytes} [Tue Aug 9 16:44:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:51.592383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d075b-c631-41c4-aeab-db8850e75fa2 service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:51.593592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-820d075b-c631-41c4-aeab-db8850e75fa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:51.687988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d075b-c631-41c4-aeab-db8850e75fa2 service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:51.730591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:51.763966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927ef8b-b298-404a-8e09-7e2769c9c8d6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33003a9bf4454f5bac2e6ad73544efc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:51.764570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927ef8b-b298-404a-8e09-7e2769c9c8d6 service glance] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:51.764846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927ef8b-b298-404a-8e09-7e2769c9c8d6 service glance] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:51.765110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927ef8b-b298-404a-8e09-7e2769c9c8d6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:51.767241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4927ef8b-b298-404a-8e09-7e2769c9c8d6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:51.905227 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927ef8b-b298-404a-8e09-7e2769c9c8d6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:51.906815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927ef8b-b298-404a-8e09-7e2769c9c8d6 service glance] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:51.927536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 694/1396] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:52.875358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98b3aa-6f3a-4367-9d59-d7a61d8fa7ab None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:52.876467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98b3aa-6f3a-4367-9d59-d7a61d8fa7ab None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:52.876467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98b3aa-6f3a-4367-9d59-d7a61d8fa7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:52.968010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-3f98b3aa-6f3a-4367-9d59-d7a61d8fa7ab None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:52.969234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3f98b3aa-6f3a-4367-9d59-d7a61d8fa7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:53.070667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 703/1397] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 16:44:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:53.123245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203d5b-d894-48e5-a43f-0f81300710b6 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:53.126863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203d5b-d894-48e5-a43f-0f81300710b6 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:53.126863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203d5b-d894-48e5-a43f-0f81300710b6 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:53.126863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 695/1398] 10.222.0.79 () {54 vars in 943 bytes} [Tue Aug 9 16:44:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:53.133471 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca116c6-2d92-4393-9778-0753a8689649 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:53.134202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca116c6-2d92-4393-9778-0753a8689649 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:53.134202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca116c6-2d92-4393-9778-0753a8689649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:53.198789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:53.217778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-fca116c6-2d92-4393-9778-0753a8689649 None None] MFA Rules not processed for user `35371b488c474b7f851af16bdad34f69`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:53.219147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-fca116c6-2d92-4393-9778-0753a8689649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:53.331401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 704/1399] 10.222.0.79 () {58 vars in 1040 bytes} [Tue Aug 9 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:53.341456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd203d5b-d894-48e5-a43f-0f81300710b6 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:53.343197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fd203d5b-d894-48e5-a43f-0f81300710b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:53.428973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:53.491205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd203d5b-d894-48e5-a43f-0f81300710b6 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:53.603347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512890f6-49ef-4465-baab-818e982ba992 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:53.603944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512890f6-49ef-4465-baab-818e982ba992 service swift] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:53.604216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512890f6-49ef-4465-baab-818e982ba992 service swift] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:53.604480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512890f6-49ef-4465-baab-818e982ba992 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:53.606251 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-512890f6-49ef-4465-baab-818e982ba992 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:53.673036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:53.703836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512890f6-49ef-4465-baab-818e982ba992 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:53.705557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512890f6-49ef-4465-baab-818e982ba992 service swift] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:53.714354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 696/1400] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 16:44:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 16:44:53.771238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e247be-6b6a-49f4-b932-8912e3e37b82 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:53.771497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e247be-6b6a-49f4-b932-8912e3e37b82 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:53.771790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e247be-6b6a-49f4-b932-8912e3e37b82 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:53.773161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 705/1401] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:53.778977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3d571-87c5-4f1c-b666-9c653304f47d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:53.778977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3d571-87c5-4f1c-b666-9c653304f47d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:53.779185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3d571-87c5-4f1c-b666-9c653304f47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:53.850328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0df3d571-87c5-4f1c-b666-9c653304f47d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:53.851549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0df3d571-87c5-4f1c-b666-9c653304f47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:53.958011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 697/1402] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:44:54.003408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e247be-6b6a-49f4-b932-8912e3e37b82 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:54.004943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e8e247be-6b6a-49f4-b932-8912e3e37b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:54.134770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:54.183271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e247be-6b6a-49f4-b932-8912e3e37b82 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:54.333360 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27226556-e0ea-4527-b5bf-269f01dd775d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:54.333988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27226556-e0ea-4527-b5bf-269f01dd775d service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:54.334281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27226556-e0ea-4527-b5bf-269f01dd775d service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:54.334531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27226556-e0ea-4527-b5bf-269f01dd775d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:54.336135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-27226556-e0ea-4527-b5bf-269f01dd775d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:54.455841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:54.476974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27226556-e0ea-4527-b5bf-269f01dd775d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:54.478908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27226556-e0ea-4527-b5bf-269f01dd775d service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:54.499339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 706/1403] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:56.852791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c274f-6ed0-4fd6-949b-f43243cdcafa None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:56.853402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c274f-6ed0-4fd6-949b-f43243cdcafa None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:56.853402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262c274f-6ed0-4fd6-949b-f43243cdcafa None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:56.855165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 698/1404] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:56.861053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc611f-6e2a-4eca-b997-b744acd8b9fc None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:56.861556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc611f-6e2a-4eca-b997-b744acd8b9fc None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:56.861660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc611f-6e2a-4eca-b997-b744acd8b9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:56.944419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-aabc611f-6e2a-4eca-b997-b744acd8b9fc None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:44:56.945731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-aabc611f-6e2a-4eca-b997-b744acd8b9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:57.057908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 707/1405] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:44:57.107644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c274f-6ed0-4fd6-949b-f43243cdcafa None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:44:57.109012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-262c274f-6ed0-4fd6-949b-f43243cdcafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:57.291592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:57.412427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262c274f-6ed0-4fd6-949b-f43243cdcafa None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:44:57.545900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49d156-e9ac-405f-b0ab-02dc886bbb8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:44:57.546528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49d156-e9ac-405f-b0ab-02dc886bbb8b service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:57.546803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49d156-e9ac-405f-b0ab-02dc886bbb8b service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:57.547055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49d156-e9ac-405f-b0ab-02dc886bbb8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:57.548665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-af49d156-e9ac-405f-b0ab-02dc886bbb8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:44:57.561303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:44:57.651894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49d156-e9ac-405f-b0ab-02dc886bbb8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:44:57.653670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49d156-e9ac-405f-b0ab-02dc886bbb8b service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:44:57.674522 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 699/1406] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:44:59.977559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5859af-c580-41c4-8315-67d1c9356696 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:59.978227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5859af-c580-41c4-8315-67d1c9356696 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:59.978227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5859af-c580-41c4-8315-67d1c9356696 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:44:59.979488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 708/1407] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:44:59.985276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bcfe8-52d0-4a1a-b586-dfab0999d199 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:44:59.985761 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bcfe8-52d0-4a1a-b586-dfab0999d199 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:44:59.985863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bcfe8-52d0-4a1a-b586-dfab0999d199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:00.069733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6f5bcfe8-52d0-4a1a-b586-dfab0999d199 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:00.070861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6f5bcfe8-52d0-4a1a-b586-dfab0999d199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:00.194559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:00.211651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 700/1408] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:00.266091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5859af-c580-41c4-8315-67d1c9356696 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:00.267537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2f5859af-c580-41c4-8315-67d1c9356696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:00.341395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:00.514165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5859af-c580-41c4-8315-67d1c9356696 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:00.673500 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa0a020-41f7-4176-883e-584edcf21d8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:00.674259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0a020-41f7-4176-883e-584edcf21d8d service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:00.674705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0a020-41f7-4176-883e-584edcf21d8d service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:00.675120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0a020-41f7-4176-883e-584edcf21d8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:00.676799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8fa0a020-41f7-4176-883e-584edcf21d8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:00.691507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:00.790804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0a020-41f7-4176-883e-584edcf21d8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:00.792757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa0a020-41f7-4176-883e-584edcf21d8d service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:00.816136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 709/1409] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:10.224868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:10.812751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=41 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:17.773978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51434d1-28c8-45d9-8675-2ac905f3c5f9 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:17.774559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51434d1-28c8-45d9-8675-2ac905f3c5f9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:17.774559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51434d1-28c8-45d9-8675-2ac905f3c5f9 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:17.775827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 701/1410] 10.222.0.79 () {54 vars in 918 bytes} [Tue Aug 9 16:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:17.780958 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf25647-6b89-4eda-a6c9-b72ee12e3036 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:17.781856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf25647-6b89-4eda-a6c9-b72ee12e3036 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:17.781856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf25647-6b89-4eda-a6c9-b72ee12e3036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:17.881565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-6cf25647-6b89-4eda-a6c9-b72ee12e3036 None None] MFA Rules not processed for user `d7bc938254514471801c88761525e883`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:17.882810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6cf25647-6b89-4eda-a6c9-b72ee12e3036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:17.993929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 710/1411] 10.222.0.79 () {58 vars in 1015 bytes} [Tue Aug 9 16:45:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:18.009940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c43bc-a6f2-45c4-8575-19db8b8d440e None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:18.010306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c43bc-a6f2-45c4-8575-19db8b8d440e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:18.010573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c43bc-a6f2-45c4-8575-19db8b8d440e None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:18.011981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 702/1412] 10.222.0.79 () {54 vars in 918 bytes} [Tue Aug 9 16:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:18.017016 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fdea0-c4b4-4a3c-914f-cf3fd7bf4bbb None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:18.017304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fdea0-c4b4-4a3c-914f-cf3fd7bf4bbb None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:18.017580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fdea0-c4b4-4a3c-914f-cf3fd7bf4bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:18.092413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:18.095987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-664fdea0-c4b4-4a3c-914f-cf3fd7bf4bbb None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:18.097255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-664fdea0-c4b4-4a3c-914f-cf3fd7bf4bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:18.219260 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 711/1413] 10.222.0.79 () {58 vars in 1015 bytes} [Tue Aug 9 16:45:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:45:20.742911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f815f2-929e-4e62-b7bf-89420773e17d None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:20.743494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f815f2-929e-4e62-b7bf-89420773e17d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:20.743494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f815f2-929e-4e62-b7bf-89420773e17d None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:20.744746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:20.750555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c707637-0ecd-40f4-9bef-e835488c0e78 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:20.751015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c707637-0ecd-40f4-9bef-e835488c0e78 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:20.751311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c707637-0ecd-40f4-9bef-e835488c0e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:20.833527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8c707637-0ecd-40f4-9bef-e835488c0e78 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:20.834782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8c707637-0ecd-40f4-9bef-e835488c0e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:20.893014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:20.940894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:20.985032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f815f2-929e-4e62-b7bf-89420773e17d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:20.986594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-36f815f2-929e-4e62-b7bf-89420773e17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:21.141168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f815f2-929e-4e62-b7bf-89420773e17d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:21.217294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:21.257603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7120036-1961-4d05-9219-258519110db5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:21.258216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120036-1961-4d05-9219-258519110db5 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:21.258486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120036-1961-4d05-9219-258519110db5 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:21.258745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7120036-1961-4d05-9219-258519110db5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:21.260330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a7120036-1961-4d05-9219-258519110db5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:21.348102 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7120036-1961-4d05-9219-258519110db5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:21.349666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7120036-1961-4d05-9219-258519110db5 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:21.367059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 704/1416] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:22.290796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c8777-f072-402e-8f4a-5348c195448b None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:22.291296 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c8777-f072-402e-8f4a-5348c195448b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:22.291403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c8777-f072-402e-8f4a-5348c195448b None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:22.292855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:22.299677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d12fe-15d4-466e-965b-5872260a0143 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:22.300183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d12fe-15d4-466e-965b-5872260a0143 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:22.300183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d12fe-15d4-466e-965b-5872260a0143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:22.379999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2b8d12fe-15d4-466e-965b-5872260a0143 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:22.381216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2b8d12fe-15d4-466e-965b-5872260a0143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:22.408615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:22.492152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:45:22.644160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c8777-f072-402e-8f4a-5348c195448b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:22.645651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7e7c8777-f072-402e-8f4a-5348c195448b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:22.791926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c8777-f072-402e-8f4a-5348c195448b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:22.834833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:22.913355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31965846-a135-4fa1-9660-6de5ff4b3434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:22.913986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31965846-a135-4fa1-9660-6de5ff4b3434 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:22.914269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31965846-a135-4fa1-9660-6de5ff4b3434 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:22.914541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31965846-a135-4fa1-9660-6de5ff4b3434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:22.916179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-31965846-a135-4fa1-9660-6de5ff4b3434 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:23.012122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31965846-a135-4fa1-9660-6de5ff4b3434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:23.013708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31965846-a135-4fa1-9660-6de5ff4b3434 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:23.033449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 714/1419] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:23.949171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165ef3f-1409-426d-b98f-4b8a8f342cb6 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:23.949720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165ef3f-1409-426d-b98f-4b8a8f342cb6 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:23.949720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165ef3f-1409-426d-b98f-4b8a8f342cb6 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:23.951031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:23.956460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381bc81-a0b3-4207-afa8-a706f178a29c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:23.956874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381bc81-a0b3-4207-afa8-a706f178a29c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:23.956984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381bc81-a0b3-4207-afa8-a706f178a29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:24.012837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:24.035914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-6381bc81-a0b3-4207-afa8-a706f178a29c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:24.037138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6381bc81-a0b3-4207-afa8-a706f178a29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:24.144117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:24.297758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165ef3f-1409-426d-b98f-4b8a8f342cb6 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:24.299353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f165ef3f-1409-426d-b98f-4b8a8f342cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:24.449576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165ef3f-1409-426d-b98f-4b8a8f342cb6 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:24.462449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:24.575855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2a387-aa3a-445b-b273-58aa7c9ca2d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:24.576465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2a387-aa3a-445b-b273-58aa7c9ca2d7 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:24.576737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2a387-aa3a-445b-b273-58aa7c9ca2d7 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:24.577002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2a387-aa3a-445b-b273-58aa7c9ca2d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:24.579256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bad2a387-aa3a-445b-b273-58aa7c9ca2d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:24.669559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2a387-aa3a-445b-b273-58aa7c9ca2d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:24.671375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2a387-aa3a-445b-b273-58aa7c9ca2d7 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:24.690991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 707/1422] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:25.612161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166af95e-8d88-45f5-99c3-210b2eacc743 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:25.612894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166af95e-8d88-45f5-99c3-210b2eacc743 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:25.612894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166af95e-8d88-45f5-99c3-210b2eacc743 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:25.614184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:25.620084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c12d-6186-4ed5-bff2-c180afe02e92 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:25.620582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c12d-6186-4ed5-bff2-c180afe02e92 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:25.620969 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778c12d-6186-4ed5-bff2-c180afe02e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:25.647711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:25.711226 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6778c12d-6186-4ed5-bff2-c180afe02e92 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:25.712359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6778c12d-6186-4ed5-bff2-c180afe02e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:25.825291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:25.977536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166af95e-8d88-45f5-99c3-210b2eacc743 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:25.979260 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-166af95e-8d88-45f5-99c3-210b2eacc743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:26.092473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:26.131533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166af95e-8d88-45f5-99c3-210b2eacc743 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:26.256215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ddba9-140e-4644-852f-8c899153083f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:26.256880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ddba9-140e-4644-852f-8c899153083f service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:26.257160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ddba9-140e-4644-852f-8c899153083f service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:26.257430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ddba9-140e-4644-852f-8c899153083f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:26.259246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-623ddba9-140e-4644-852f-8c899153083f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:26.343157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:26.358238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ddba9-140e-4644-852f-8c899153083f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:26.359916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ddba9-140e-4644-852f-8c899153083f service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:26.378511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 717/1425] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:27.270001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70908418-330f-4e82-ad0e-e7f876e95a7e None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:27.270616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70908418-330f-4e82-ad0e-e7f876e95a7e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:27.270616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70908418-330f-4e82-ad0e-e7f876e95a7e None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:27.271894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:27.277265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf04d9-d37b-4aeb-9915-55d31d533a6b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:27.277720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf04d9-d37b-4aeb-9915-55d31d533a6b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:27.277830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf04d9-d37b-4aeb-9915-55d31d533a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:27.353552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-e0cf04d9-d37b-4aeb-9915-55d31d533a6b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:27.354768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e0cf04d9-d37b-4aeb-9915-55d31d533a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:27.451886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:27.601877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70908418-330f-4e82-ad0e-e7f876e95a7e None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:27.603466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-70908418-330f-4e82-ad0e-e7f876e95a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:27.692004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:27.756207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70908418-330f-4e82-ad0e-e7f876e95a7e None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:27.872463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120317c8-4280-41d0-bac4-61bf93282a9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:27.872970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120317c8-4280-41d0-bac4-61bf93282a9a service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:27.873241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120317c8-4280-41d0-bac4-61bf93282a9a service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:27.873501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120317c8-4280-41d0-bac4-61bf93282a9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:27.875789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-120317c8-4280-41d0-bac4-61bf93282a9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:27.924617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:27.966039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120317c8-4280-41d0-bac4-61bf93282a9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:27.967657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120317c8-4280-41d0-bac4-61bf93282a9a service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:27.986585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 710/1428] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:45:28.874160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34e9d4-7b79-4697-bb2a-640b52ee4ac9 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:28.874733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34e9d4-7b79-4697-bb2a-640b52ee4ac9 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:28.874733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34e9d4-7b79-4697-bb2a-640b52ee4ac9 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:28.876011 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:28.881111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ead19-de6b-4c66-9b85-52a2390da894 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:28.881532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ead19-de6b-4c66-9b85-52a2390da894 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:28.881645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ead19-de6b-4c66-9b85-52a2390da894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:28.961720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ac7ead19-de6b-4c66-9b85-52a2390da894 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:28.962390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ac7ead19-de6b-4c66-9b85-52a2390da894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:29.060260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:29.215544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34e9d4-7b79-4697-bb2a-640b52ee4ac9 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:29.216978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9c34e9d4-7b79-4697-bb2a-640b52ee4ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:29.388787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:29.488457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34e9d4-7b79-4697-bb2a-640b52ee4ac9 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:29.605598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35013d3-cc9b-467a-9357-2fae57c902fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:29.606218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35013d3-cc9b-467a-9357-2fae57c902fe service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:29.606609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35013d3-cc9b-467a-9357-2fae57c902fe service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:29.606907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35013d3-cc9b-467a-9357-2fae57c902fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:29.608341 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d35013d3-cc9b-467a-9357-2fae57c902fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:29.615450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:29.696896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35013d3-cc9b-467a-9357-2fae57c902fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:29.698807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35013d3-cc9b-467a-9357-2fae57c902fe service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:29.718110 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 720/1431] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:30.644549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721204a0-5e42-46cb-bf7b-4f7b82dfaf41 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:30.645142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721204a0-5e42-46cb-bf7b-4f7b82dfaf41 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:30.645142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721204a0-5e42-46cb-bf7b-4f7b82dfaf41 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:30.646430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:30.651540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b7f81-e9c5-41ac-a821-dc440b887af3 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:30.652038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b7f81-e9c5-41ac-a821-dc440b887af3 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:30.652154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b7f81-e9c5-41ac-a821-dc440b887af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:30.731663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a28b7f81-e9c5-41ac-a821-dc440b887af3 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:30.732879 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a28b7f81-e9c5-41ac-a821-dc440b887af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:30.803343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:30.833018 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:30.986191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721204a0-5e42-46cb-bf7b-4f7b82dfaf41 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:30.987982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-721204a0-5e42-46cb-bf7b-4f7b82dfaf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:31.005680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:31.136083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721204a0-5e42-46cb-bf7b-4f7b82dfaf41 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:31.226261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:31.253350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f26837-76dd-461d-a664-92104dec55e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:31.253943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f26837-76dd-461d-a664-92104dec55e0 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:31.254236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f26837-76dd-461d-a664-92104dec55e0 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:31.254497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f26837-76dd-461d-a664-92104dec55e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:31.256058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-02f26837-76dd-461d-a664-92104dec55e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:31.345852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f26837-76dd-461d-a664-92104dec55e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:31.347582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f26837-76dd-461d-a664-92104dec55e0 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:31.365669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 713/1434] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:32.274415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c003f-1e1c-4fe1-b325-0f26796ead29 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:32.275159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c003f-1e1c-4fe1-b325-0f26796ead29 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:32.275159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c003f-1e1c-4fe1-b325-0f26796ead29 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:32.276309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:32.282503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa9346-7a96-4cf4-8699-2ef780805f8e None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:32.282503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa9346-7a96-4cf4-8699-2ef780805f8e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:32.283086 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa9346-7a96-4cf4-8699-2ef780805f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:32.361262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-49fa9346-7a96-4cf4-8699-2ef780805f8e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:32.362507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-49fa9346-7a96-4cf4-8699-2ef780805f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:32.404189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:32.466499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:45:32.623667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c003f-1e1c-4fe1-b325-0f26796ead29 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:32.625731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-843c003f-1e1c-4fe1-b325-0f26796ead29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:32.776571 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c003f-1e1c-4fe1-b325-0f26796ead29 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:32.833575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:32.895285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e1ca3-6570-4bbd-9fb3-0e03ed4bad0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:32.895878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1ca3-6570-4bbd-9fb3-0e03ed4bad0c service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:32.896150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1ca3-6570-4bbd-9fb3-0e03ed4bad0c service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:32.896416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e1ca3-6570-4bbd-9fb3-0e03ed4bad0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:32.898054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-636e1ca3-6570-4bbd-9fb3-0e03ed4bad0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:32.991708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e1ca3-6570-4bbd-9fb3-0e03ed4bad0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:32.993320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e1ca3-6570-4bbd-9fb3-0e03ed4bad0c service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:33.012678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 723/1437] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:33.941002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcfce3-17a9-43bc-a796-5d8e5230f591 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:33.941546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcfce3-17a9-43bc-a796-5d8e5230f591 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:33.941546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcfce3-17a9-43bc-a796-5d8e5230f591 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:33.942907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:33.948631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8abe0-37e5-4881-8fb7-260f068ebb48 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:33.949303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8abe0-37e5-4881-8fb7-260f068ebb48 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:33.949303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8abe0-37e5-4881-8fb7-260f068ebb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:34.030618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a0c8abe0-37e5-4881-8fb7-260f068ebb48 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:34.032446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a0c8abe0-37e5-4881-8fb7-260f068ebb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:34.034665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:34.135881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:34.289413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcfce3-17a9-43bc-a796-5d8e5230f591 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:34.291131 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a7dcfce3-17a9-43bc-a796-5d8e5230f591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:34.440797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcfce3-17a9-43bc-a796-5d8e5230f591 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:34.471464 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:34.566429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734da351-1608-470d-a3eb-9b935798d449 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:34.566949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da351-1608-470d-a3eb-9b935798d449 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:34.567224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da351-1608-470d-a3eb-9b935798d449 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:34.567489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da351-1608-470d-a3eb-9b935798d449 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:34.570283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-734da351-1608-470d-a3eb-9b935798d449 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:34.665591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734da351-1608-470d-a3eb-9b935798d449 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:34.667211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734da351-1608-470d-a3eb-9b935798d449 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:34.686573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 716/1440] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:36.626246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391589b-b331-4027-99b1-3c4ecb8593cb None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:36.626818 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391589b-b331-4027-99b1-3c4ecb8593cb None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:36.626818 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391589b-b331-4027-99b1-3c4ecb8593cb None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:36.628195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:36.634321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc1da1-239d-4c80-a45d-094c37b1efaa None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:36.635121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc1da1-239d-4c80-a45d-094c37b1efaa None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:36.635121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc1da1-239d-4c80-a45d-094c37b1efaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:36.681996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:36.708561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-8abc1da1-239d-4c80-a45d-094c37b1efaa None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:36.709847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8abc1da1-239d-4c80-a45d-094c37b1efaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:36.811633 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:36.852647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391589b-b331-4027-99b1-3c4ecb8593cb None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:36.854190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5391589b-b331-4027-99b1-3c4ecb8593cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:36.960051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391589b-b331-4027-99b1-3c4ecb8593cb None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:36.993466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:37.038402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68842107-8051-49d7-8a31-85b8d1797313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:37.039002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68842107-8051-49d7-8a31-85b8d1797313 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:37.039278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68842107-8051-49d7-8a31-85b8d1797313 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:37.039544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68842107-8051-49d7-8a31-85b8d1797313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:37.041145 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-68842107-8051-49d7-8a31-85b8d1797313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:37.166898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68842107-8051-49d7-8a31-85b8d1797313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:37.168274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68842107-8051-49d7-8a31-85b8d1797313 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:37.188009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 726/1443] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:39.418166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcec8f0-a285-4553-ad10-57f12176ae35 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:39.418804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcec8f0-a285-4553-ad10-57f12176ae35 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:39.418804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcec8f0-a285-4553-ad10-57f12176ae35 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:39.420060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:39.426393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed596ba4-86ae-4db8-be91-c306197e0161 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:39.426878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed596ba4-86ae-4db8-be91-c306197e0161 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:39.427008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed596ba4-86ae-4db8-be91-c306197e0161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:39.493525 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ed596ba4-86ae-4db8-be91-c306197e0161 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:39.494774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ed596ba4-86ae-4db8-be91-c306197e0161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:39.507887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:39.595564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:39.633881 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcec8f0-a285-4553-ad10-57f12176ae35 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:39.635435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-adcec8f0-a285-4553-ad10-57f12176ae35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:39.747210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcec8f0-a285-4553-ad10-57f12176ae35 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:39.759476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:39.825223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e8b32-62b9-4dca-91c0-71867bdcda28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:39.825827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e8b32-62b9-4dca-91c0-71867bdcda28 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:39.826202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e8b32-62b9-4dca-91c0-71867bdcda28 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:39.826402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e8b32-62b9-4dca-91c0-71867bdcda28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:39.828906 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-231e8b32-62b9-4dca-91c0-71867bdcda28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:39.947438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e8b32-62b9-4dca-91c0-71867bdcda28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:39.948875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e8b32-62b9-4dca-91c0-71867bdcda28 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:39.968808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 719/1446] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:42.376884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2d11-0766-481d-bca1-6374ac4973f5 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:42.377455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2d11-0766-481d-bca1-6374ac4973f5 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:42.377455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e2d11-0766-481d-bca1-6374ac4973f5 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:42.378819 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:42.384348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee77af1-8562-4377-8fbd-c615ae64ce32 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:42.384794 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee77af1-8562-4377-8fbd-c615ae64ce32 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:42.385132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee77af1-8562-4377-8fbd-c615ae64ce32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:42.439108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:42.459189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2ee77af1-8562-4377-8fbd-c615ae64ce32 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:42.460420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2ee77af1-8562-4377-8fbd-c615ae64ce32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:42.558549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:42.600128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e2d11-0766-481d-bca1-6374ac4973f5 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:42.601710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-063e2d11-0766-481d-bca1-6374ac4973f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:42.704855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e2d11-0766-481d-bca1-6374ac4973f5 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:42.743801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:42.779835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848efcc-0b0b-4a50-a73b-2a597c3cf9de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:42.780428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848efcc-0b0b-4a50-a73b-2a597c3cf9de service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:42.780701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848efcc-0b0b-4a50-a73b-2a597c3cf9de service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:42.780968 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848efcc-0b0b-4a50-a73b-2a597c3cf9de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:42.782577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e848efcc-0b0b-4a50-a73b-2a597c3cf9de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:42.898422 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848efcc-0b0b-4a50-a73b-2a597c3cf9de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:42.899856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848efcc-0b0b-4a50-a73b-2a597c3cf9de service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:42.918393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 729/1449] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:45.103060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5565e8e-fb40-4069-a610-704d51ba0d2d None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:45.103632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5565e8e-fb40-4069-a610-704d51ba0d2d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:45.103632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5565e8e-fb40-4069-a610-704d51ba0d2d None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:45.104922 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:45:45.110414 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0903e7d-9710-4725-a321-b334054b4b0b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:45.110879 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0903e7d-9710-4725-a321-b334054b4b0b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:45.110879 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0903e7d-9710-4725-a321-b334054b4b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:45.176396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-e0903e7d-9710-4725-a321-b334054b4b0b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:45.177698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e0903e7d-9710-4725-a321-b334054b4b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:45.199335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:45.282515 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:45.322495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5565e8e-fb40-4069-a610-704d51ba0d2d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:45.324118 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b5565e8e-fb40-4069-a610-704d51ba0d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:45.428384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5565e8e-fb40-4069-a610-704d51ba0d2d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:45.448408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:45.504220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5933a-e36b-49b7-a9d1-a158b8ab0187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:45.505252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5933a-e36b-49b7-a9d1-a158b8ab0187 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:45.505516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5933a-e36b-49b7-a9d1-a158b8ab0187 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:45.505774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5933a-e36b-49b7-a9d1-a158b8ab0187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:45.507384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-aec5933a-e36b-49b7-a9d1-a158b8ab0187 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:45.614596 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5933a-e36b-49b7-a9d1-a158b8ab0187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:45.616002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5933a-e36b-49b7-a9d1-a158b8ab0187 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:45.633967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 722/1452] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:47.712856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229b4fe-a21d-4889-925d-1ad446d4b37d None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:47.713426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229b4fe-a21d-4889-925d-1ad446d4b37d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:47.713426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229b4fe-a21d-4889-925d-1ad446d4b37d None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:47.715015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:47.720540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca06f9b-f2c9-4b18-82cc-191357390a8a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:47.721045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca06f9b-f2c9-4b18-82cc-191357390a8a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:47.721045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca06f9b-f2c9-4b18-82cc-191357390a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:47.785830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:47.788439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6ca06f9b-f2c9-4b18-82cc-191357390a8a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:47.789774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6ca06f9b-f2c9-4b18-82cc-191357390a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:47.904162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:47.943549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229b4fe-a21d-4889-925d-1ad446d4b37d None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:47.945116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2229b4fe-a21d-4889-925d-1ad446d4b37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:48.055255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229b4fe-a21d-4889-925d-1ad446d4b37d None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:48.105360 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:48.132820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76764b-da85-4afb-bdcf-b85540da6dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:48.133417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76764b-da85-4afb-bdcf-b85540da6dcb service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:48.133682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76764b-da85-4afb-bdcf-b85540da6dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:48.133944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76764b-da85-4afb-bdcf-b85540da6dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:48.135606 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2d76764b-da85-4afb-bdcf-b85540da6dcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:48.259973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76764b-da85-4afb-bdcf-b85540da6dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:48.261463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76764b-da85-4afb-bdcf-b85540da6dcb service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:48.281799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 732/1455] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:50.433177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dbbf3-e72d-4b21-a53e-74138c4004d4 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:50.439043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dbbf3-e72d-4b21-a53e-74138c4004d4 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:50.439043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9dbbf3-e72d-4b21-a53e-74138c4004d4 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:50.439043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:50.441421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908778f-86fc-4db0-a281-a099d329e82a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:50.441853 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908778f-86fc-4db0-a281-a099d329e82a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:50.441984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908778f-86fc-4db0-a281-a099d329e82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:50.519516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-2908778f-86fc-4db0-a281-a099d329e82a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:50.520724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2908778f-86fc-4db0-a281-a099d329e82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:50.551600 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:50.617871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:50.656116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9dbbf3-e72d-4b21-a53e-74138c4004d4 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:50.657622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3c9dbbf3-e72d-4b21-a53e-74138c4004d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:50.760223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9dbbf3-e72d-4b21-a53e-74138c4004d4 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:50.787968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:50.833765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ee97b-492c-48f4-a138-57ae3e9ab57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:50.834393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ee97b-492c-48f4-a138-57ae3e9ab57b service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:50.834664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ee97b-492c-48f4-a138-57ae3e9ab57b service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:50.834926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ee97b-492c-48f4-a138-57ae3e9ab57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:50.836493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-113ee97b-492c-48f4-a138-57ae3e9ab57b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:50.944592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ee97b-492c-48f4-a138-57ae3e9ab57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:50.946067 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ee97b-492c-48f4-a138-57ae3e9ab57b service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:50.966258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 725/1458] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:53.018519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277212c-7b19-4e93-93cd-503306248933 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:53.019108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277212c-7b19-4e93-93cd-503306248933 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:53.019108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277212c-7b19-4e93-93cd-503306248933 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:53.020721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:53.026343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5691f1b-2f07-4490-bd89-ab029484ce54 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:53.026812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5691f1b-2f07-4490-bd89-ab029484ce54 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:53.026812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5691f1b-2f07-4490-bd89-ab029484ce54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:53.095517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d5691f1b-2f07-4490-bd89-ab029484ce54 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:53.096730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d5691f1b-2f07-4490-bd89-ab029484ce54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:53.105534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:53.196461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:45:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:53.235614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277212c-7b19-4e93-93cd-503306248933 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:53.237140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c277212c-7b19-4e93-93cd-503306248933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:53.341406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277212c-7b19-4e93-93cd-503306248933 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:53.399026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:53.417016 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7051a5-28d1-4481-8b52-1581bfcc5152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:53.417607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7051a5-28d1-4481-8b52-1581bfcc5152 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:53.417883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7051a5-28d1-4481-8b52-1581bfcc5152 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:53.418174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7051a5-28d1-4481-8b52-1581bfcc5152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:53.419745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-af7051a5-28d1-4481-8b52-1581bfcc5152 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:53.528054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7051a5-28d1-4481-8b52-1581bfcc5152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:53.529506 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7051a5-28d1-4481-8b52-1581bfcc5152 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:53.548561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 735/1461] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:55.618926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fdbd9-6336-4919-9cad-c319367a5465 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:55.619525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fdbd9-6336-4919-9cad-c319367a5465 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:55.619525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fdbd9-6336-4919-9cad-c319367a5465 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:55.620788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 09 16:45:55.626426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aaa81-8b2f-4e5a-8327-724672219e87 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:55.626910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aaa81-8b2f-4e5a-8327-724672219e87 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:55.627026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893aaa81-8b2f-4e5a-8327-724672219e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:55.693138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-893aaa81-8b2f-4e5a-8327-724672219e87 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:55.695307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-893aaa81-8b2f-4e5a-8327-724672219e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:55.739029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:55.799181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:55.838666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fdbd9-6336-4919-9cad-c319367a5465 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:55.840222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b91fdbd9-6336-4919-9cad-c319367a5465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:55.956801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fdbd9-6336-4919-9cad-c319367a5465 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:55.996535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:56.037071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e18832-0a73-4988-92fd-70b4e1450223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:56.038142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e18832-0a73-4988-92fd-70b4e1450223 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:56.038555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e18832-0a73-4988-92fd-70b4e1450223 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:56.038920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e18832-0a73-4988-92fd-70b4e1450223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:56.040630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-75e18832-0a73-4988-92fd-70b4e1450223 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:56.156284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e18832-0a73-4988-92fd-70b4e1450223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:56.157725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e18832-0a73-4988-92fd-70b4e1450223 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:56.177543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 728/1464] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:45:58.281409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1d250-d50c-4c87-8eab-d369b504d494 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:58.282096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1d250-d50c-4c87-8eab-d369b504d494 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:58.282096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1d250-d50c-4c87-8eab-d369b504d494 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:58.283328 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:45:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:45:58.288550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59491a20-688c-4d42-9d01-b30764918903 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:58.289070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59491a20-688c-4d42-9d01-b30764918903 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:58.289070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59491a20-688c-4d42-9d01-b30764918903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:58.355174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-59491a20-688c-4d42-9d01-b30764918903 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:45:58.356398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-59491a20-688c-4d42-9d01-b30764918903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:58.374486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:58.455527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:45:58.492725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1d250-d50c-4c87-8eab-d369b504d494 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:45:58.494322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5ec1d250-d50c-4c87-8eab-d369b504d494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:58.602453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1d250-d50c-4c87-8eab-d369b504d494 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:45:58.668553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:45:58.675558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc816b-e2de-4837-895c-eda43b2d10fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:45:58.679080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc816b-e2de-4837-895c-eda43b2d10fa service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:45:58.679080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc816b-e2de-4837-895c-eda43b2d10fa service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:45:58.679080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc816b-e2de-4837-895c-eda43b2d10fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:45:58.679080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-80bc816b-e2de-4837-895c-eda43b2d10fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:45:58.786590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc816b-e2de-4837-895c-eda43b2d10fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:45:58.787996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc816b-e2de-4837-895c-eda43b2d10fa service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:45:58.806961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 738/1467] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:45:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:46:00.876099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c03d8d-d781-4bd5-a6ea-ee5688414e72 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:00.876636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c03d8d-d781-4bd5-a6ea-ee5688414e72 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:00.876636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c03d8d-d781-4bd5-a6ea-ee5688414e72 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:00.877871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:46:00.883086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8521e1-bc32-479c-bb33-2649752dae9f None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:00.883536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8521e1-bc32-479c-bb33-2649752dae9f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:00.883634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8521e1-bc32-479c-bb33-2649752dae9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:00.949068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-2d8521e1-bc32-479c-bb33-2649752dae9f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:46:00.950345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2d8521e1-bc32-479c-bb33-2649752dae9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:00.997926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:01.046769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:46:01.084987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c03d8d-d781-4bd5-a6ea-ee5688414e72 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:46:01.086549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-58c03d8d-d781-4bd5-a6ea-ee5688414e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:01.189707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c03d8d-d781-4bd5-a6ea-ee5688414e72 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:46:01.238188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:01.263835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24292a19-93aa-41ab-a223-0e557caf9c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:46:01.264435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24292a19-93aa-41ab-a223-0e557caf9c45 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:01.264710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24292a19-93aa-41ab-a223-0e557caf9c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:01.264978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24292a19-93aa-41ab-a223-0e557caf9c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:01.266597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-24292a19-93aa-41ab-a223-0e557caf9c45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:01.374238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24292a19-93aa-41ab-a223-0e557caf9c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:46:01.375660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24292a19-93aa-41ab-a223-0e557caf9c45 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:46:01.394883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 731/1470] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:46:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:46:11.047926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=22 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:11.392703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=66 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:21.100163 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98008981-95e6-42d0-8987-dcdcaccd7a6f None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:21.100714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98008981-95e6-42d0-8987-dcdcaccd7a6f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:21.100714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98008981-95e6-42d0-8987-dcdcaccd7a6f None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:21.102109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 740/1471] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:46:21.107434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d214e-94ee-4142-a2b7-f92d1be86887 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:21.107888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d214e-94ee-4142-a2b7-f92d1be86887 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:21.107991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d214e-94ee-4142-a2b7-f92d1be86887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:21.182203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-925d214e-94ee-4142-a2b7-f92d1be86887 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:46:21.183413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-925d214e-94ee-4142-a2b7-f92d1be86887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:21.280081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 732/1472] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:46:21.371565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd6f6c-97a1-49ff-bb57-87859b7ff529 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:21.371840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd6f6c-97a1-49ff-bb57-87859b7ff529 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:21.372608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccd6f6c-97a1-49ff-bb57-87859b7ff529 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:21.374025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 741/1473] 10.222.0.79 () {54 vars in 967 bytes} [Tue Aug 9 16:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:46:21.379381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9ec39-316e-409a-b633-8ac615b3fc42 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:21.379663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9ec39-316e-409a-b633-8ac615b3fc42 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:21.379934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9ec39-316e-409a-b633-8ac615b3fc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:21.464242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-e9a9ec39-316e-409a-b633-8ac615b3fc42 None None] MFA Rules not processed for user `33003a9bf4454f5bac2e6ad73544efc4`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:46:21.465476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e9a9ec39-316e-409a-b633-8ac615b3fc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:21.486803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:21.548622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 733/1474] 10.222.0.79 () {58 vars in 1064 bytes} [Tue Aug 9 16:46:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:46:21.556405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd6f6c-97a1-49ff-bb57-87859b7ff529 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:46:21.557935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bccd6f6c-97a1-49ff-bb57-87859b7ff529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:21.672183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccd6f6c-97a1-49ff-bb57-87859b7ff529 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:46:21.747888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cc504-4bc9-44f4-90f3-b43bcafce92b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33003a9bf4454f5bac2e6ad73544efc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:46:21.748477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cc504-4bc9-44f4-90f3-b43bcafce92b service glance] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:21.748748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cc504-4bc9-44f4-90f3-b43bcafce92b service glance] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:21.749015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cc504-4bc9-44f4-90f3-b43bcafce92b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:21.750631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-327cc504-4bc9-44f4-90f3-b43bcafce92b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:21.804964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:21.860536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cc504-4bc9-44f4-90f3-b43bcafce92b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:46:21.862118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cc504-4bc9-44f4-90f3-b43bcafce92b service glance] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:46:21.882855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 742/1475] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:46:24.003164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c3aa9-7a9c-4210-b2c3-4ff2a8e49c88 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:24.003758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c3aa9-7a9c-4210-b2c3-4ff2a8e49c88 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:24.003758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c3aa9-7a9c-4210-b2c3-4ff2a8e49c88 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:24.005242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 734/1476] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:46:24.010716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f2148-d4a2-4bf1-9b37-3605cb90b1c4 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:24.011184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f2148-d4a2-4bf1-9b37-3605cb90b1c4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:24.011290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f2148-d4a2-4bf1-9b37-3605cb90b1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:24.076211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-072f2148-d4a2-4bf1-9b37-3605cb90b1c4 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:46:24.077402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-072f2148-d4a2-4bf1-9b37-3605cb90b1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:24.173702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 743/1477] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:46:24.197054 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c3aa9-7a9c-4210-b2c3-4ff2a8e49c88 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:46:24.198603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9f7c3aa9-7a9c-4210-b2c3-4ff2a8e49c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:24.323561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c3aa9-7a9c-4210-b2c3-4ff2a8e49c88 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:46:24.374632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:24.420806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17cbde-0cc7-4d73-a594-0a1a1a908093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:46:24.421390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cbde-0cc7-4d73-a594-0a1a1a908093 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:24.421661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cbde-0cc7-4d73-a594-0a1a1a908093 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:24.421919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cbde-0cc7-4d73-a594-0a1a1a908093 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:24.422568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17cbde-0cc7-4d73-a594-0a1a1a908093 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:46:24.424364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: WARNING py.warnings [None req-ba17cbde-0cc7-4d73-a594-0a1a1a908093 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:46:24.424364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: warnings.warn(msg) Aug 09 16:46:24.424364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: Aug 09 16:46:24.425414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17cbde-0cc7-4d73-a594-0a1a1a908093 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:46:24.443847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 735/1478] 10.222.0.79 () {56 vars in 1108 bytes} [Tue Aug 9 16:46:24 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:46:26.499137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dc2f8-dc4f-45a1-b3f4-2f7b0d30a654 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:26.499712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dc2f8-dc4f-45a1-b3f4-2f7b0d30a654 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:26.499712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5dc2f8-dc4f-45a1-b3f4-2f7b0d30a654 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:26.501024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 744/1479] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:46:26.506353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef57af6-0d8e-4fed-bad9-c04931b29ebf None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:26.506871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef57af6-0d8e-4fed-bad9-c04931b29ebf None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:26.506871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef57af6-0d8e-4fed-bad9-c04931b29ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:26.576181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-eef57af6-0d8e-4fed-bad9-c04931b29ebf None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:46:26.577400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-eef57af6-0d8e-4fed-bad9-c04931b29ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:26.671345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 736/1480] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:46:26.715850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dc2f8-dc4f-45a1-b3f4-2f7b0d30a654 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:46:26.717343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-db5dc2f8-dc4f-45a1-b3f4-2f7b0d30a654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:26.726927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:26.863050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5dc2f8-dc4f-45a1-b3f4-2f7b0d30a654 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:46:26.945191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:26.978408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebafebc-9f43-4dc4-a905-23ead8e99510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:46:26.979014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebafebc-9f43-4dc4-a905-23ead8e99510 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:26.979291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebafebc-9f43-4dc4-a905-23ead8e99510 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:26.979561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebafebc-9f43-4dc4-a905-23ead8e99510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:26.981696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6ebafebc-9f43-4dc4-a905-23ead8e99510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:27.075155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebafebc-9f43-4dc4-a905-23ead8e99510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:46:27.076744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebafebc-9f43-4dc4-a905-23ead8e99510 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:46:27.094172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 745/1481] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:46:29.326328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225758e-d5f9-4b64-8b76-9104f0cd3bb8 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:29.326908 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225758e-d5f9-4b64-8b76-9104f0cd3bb8 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:29.326908 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225758e-d5f9-4b64-8b76-9104f0cd3bb8 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:29.328194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 737/1482] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:46:29.333415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825976-a9dc-4102-a5b3-7444c3080e27 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:29.333836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825976-a9dc-4102-a5b3-7444c3080e27 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:29.333981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09825976-a9dc-4102-a5b3-7444c3080e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:29.407640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-09825976-a9dc-4102-a5b3-7444c3080e27 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:46:29.408850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-09825976-a9dc-4102-a5b3-7444c3080e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:29.454852 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:29.507514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 746/1483] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:46:29.544721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225758e-d5f9-4b64-8b76-9104f0cd3bb8 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:46:29.546299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8225758e-d5f9-4b64-8b76-9104f0cd3bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:29.570093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:29.650297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225758e-d5f9-4b64-8b76-9104f0cd3bb8 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:46:29.725010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a508b3d0-bfc7-489e-96ac-2f263c99c142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:46:29.725608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b3d0-bfc7-489e-96ac-2f263c99c142 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:29.725880 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b3d0-bfc7-489e-96ac-2f263c99c142 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:29.726177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a508b3d0-bfc7-489e-96ac-2f263c99c142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:29.727773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a508b3d0-bfc7-489e-96ac-2f263c99c142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:29.813459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:29.850942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508b3d0-bfc7-489e-96ac-2f263c99c142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:46:29.852346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a508b3d0-bfc7-489e-96ac-2f263c99c142 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:46:29.871892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 738/1484] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:46:31.964211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c140d26-f1a1-40f4-b132-a19aaf4ef214 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:31.964824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c140d26-f1a1-40f4-b132-a19aaf4ef214 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:31.964824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c140d26-f1a1-40f4-b132-a19aaf4ef214 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:31.966711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 747/1485] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:46:31.971889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e84a02-244d-430b-b9b7-8e0e4b8726ae None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:31.972304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e84a02-244d-430b-b9b7-8e0e4b8726ae None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:31.972410 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e84a02-244d-430b-b9b7-8e0e4b8726ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:32.047549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-11e84a02-244d-430b-b9b7-8e0e4b8726ae None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:46:32.048767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-11e84a02-244d-430b-b9b7-8e0e4b8726ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:32.144698 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 739/1486] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:46:32.183756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c140d26-f1a1-40f4-b132-a19aaf4ef214 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:46:32.185544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7c140d26-f1a1-40f4-b132-a19aaf4ef214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:32.296493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c140d26-f1a1-40f4-b132-a19aaf4ef214 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:46:32.378680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:32.380583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cc09e-f627-45f6-bab8-10924ef5615f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:46:32.381198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cc09e-f627-45f6-bab8-10924ef5615f service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:32.381473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cc09e-f627-45f6-bab8-10924ef5615f service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:32.381737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cc09e-f627-45f6-bab8-10924ef5615f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:32.383418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-437cc09e-f627-45f6-bab8-10924ef5615f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:32.507184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cc09e-f627-45f6-bab8-10924ef5615f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:46:32.508663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cc09e-f627-45f6-bab8-10924ef5615f service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:46:32.527674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 748/1487] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:46:34.733028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462e0c-266f-493f-9d03-3b112fecd0e1 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:34.733618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462e0c-266f-493f-9d03-3b112fecd0e1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:34.733618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92462e0c-266f-493f-9d03-3b112fecd0e1 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:34.735262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 740/1488] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:46:34.740854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaeb58-9c03-46ea-91b2-90e852cdee69 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:34.741405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaeb58-9c03-46ea-91b2-90e852cdee69 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:34.741405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfaeb58-9c03-46ea-91b2-90e852cdee69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:34.814159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-cbfaeb58-9c03-46ea-91b2-90e852cdee69 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:46:34.815311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cbfaeb58-9c03-46ea-91b2-90e852cdee69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:34.874280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:34.919370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 749/1489] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:46:34.967154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462e0c-266f-493f-9d03-3b112fecd0e1 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:46:34.968734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-92462e0c-266f-493f-9d03-3b112fecd0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:35.019215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:35.089463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92462e0c-266f-493f-9d03-3b112fecd0e1 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:46:35.168429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2142b7-273f-4532-98e9-685b0a7d3fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:46:35.169026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2142b7-273f-4532-98e9-685b0a7d3fdc service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:35.169304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2142b7-273f-4532-98e9-685b0a7d3fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:35.169574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2142b7-273f-4532-98e9-685b0a7d3fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:35.171233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fa2142b7-273f-4532-98e9-685b0a7d3fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:35.253854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:35.287688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2142b7-273f-4532-98e9-685b0a7d3fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:46:35.289118 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2142b7-273f-4532-98e9-685b0a7d3fdc service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:46:35.307955 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 741/1490] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:46:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:46:37.793877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d980bc-c1a3-4e75-8688-884ae6900c80 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:37.794416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d980bc-c1a3-4e75-8688-884ae6900c80 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:37.794416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d980bc-c1a3-4e75-8688-884ae6900c80 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:37.795758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 750/1491] 10.222.0.79 () {54 vars in 927 bytes} [Tue Aug 9 16:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:46:37.801272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f7b7-5db2-4a57-b3e9-a18e9fbece7c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:37.801708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f7b7-5db2-4a57-b3e9-a18e9fbece7c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:37.801805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a0f7b7-5db2-4a57-b3e9-a18e9fbece7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:37.868162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d5a0f7b7-5db2-4a57-b3e9-a18e9fbece7c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:46:37.869381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d5a0f7b7-5db2-4a57-b3e9-a18e9fbece7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:37.968867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 742/1492] 10.222.0.79 () {58 vars in 1024 bytes} [Tue Aug 9 16:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:46:38.008479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d980bc-c1a3-4e75-8688-884ae6900c80 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:46:38.010281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-92d980bc-c1a3-4e75-8688-884ae6900c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:38.119337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d980bc-c1a3-4e75-8688-884ae6900c80 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:46:38.188367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbc1c3-3e5e-490a-8d34-8ad71318d3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:46:38.188960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbc1c3-3e5e-490a-8d34-8ad71318d3fb service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:46:38.189237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbc1c3-3e5e-490a-8d34-8ad71318d3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:46:38.189506 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbc1c3-3e5e-490a-8d34-8ad71318d3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:46:38.191130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4bcbc1c3-3e5e-490a-8d34-8ad71318d3fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:46:38.201042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:38.299198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbc1c3-3e5e-490a-8d34-8ad71318d3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:46:38.300626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbc1c3-3e5e-490a-8d34-8ad71318d3fb service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:46:38.319235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 751/1493] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:46:47.967079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=82 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:46:48.316945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=50 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:47:59.599939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e95ac-ace2-44d2-8457-37778132e821 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:47:59.600534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e95ac-ace2-44d2-8457-37778132e821 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:47:59.600534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e95ac-ace2-44d2-8457-37778132e821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:47:59.676062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-144e95ac-ace2-44d2-8457-37778132e821 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:47:59.677271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-144e95ac-ace2-44d2-8457-37778132e821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:47:59.773355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 743/1494] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:47:59.781728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590da8c-432b-47e9-a2db-d92ac2f4753b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:47:59.782207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590da8c-432b-47e9-a2db-d92ac2f4753b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:47:59.782300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2590da8c-432b-47e9-a2db-d92ac2f4753b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:47:59.866842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-2590da8c-432b-47e9-a2db-d92ac2f4753b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:47:59.868213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2590da8c-432b-47e9-a2db-d92ac2f4753b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:47:59.972054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 752/1495] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:47:59.977004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e95ac-ace2-44d2-8457-37778132e821 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:47:59.978349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-144e95ac-ace2-44d2-8457-37778132e821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:47:59.991121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e95ac-ace2-44d2-8457-37778132e821 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:48:00.053428 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:00.087844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:48:00.088446 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:00.088723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:00.088991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:00.089627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:48:00.089924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:48:00.090951 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: WARNING py.warnings [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:48:00.090951 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: warnings.warn(msg) Aug 09 16:48:00.090951 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: Aug 09 16:48:00.091831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:48:00.103445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 744/1496] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:48:00.112378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590da8c-432b-47e9-a2db-d92ac2f4753b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:48:00.121553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2590da8c-432b-47e9-a2db-d92ac2f4753b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:48:00.185265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:00.220068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba548e73-e8c9-411f-ab6d-1f78e488e8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:48:00.220689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba548e73-e8c9-411f-ab6d-1f78e488e8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:00.220985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba548e73-e8c9-411f-ab6d-1f78e488e8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:00.221250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba548e73-e8c9-411f-ab6d-1f78e488e8c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:00.221887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba548e73-e8c9-411f-ab6d-1f78e488e8c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:48:00.223636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: WARNING py.warnings [None req-ba548e73-e8c9-411f-ab6d-1f78e488e8c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:48:00.223636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: warnings.warn(msg) Aug 09 16:48:00.223636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: Aug 09 16:48:00.224820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba548e73-e8c9-411f-ab6d-1f78e488e8c4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:48:00.367218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 753/1497] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:48:00 2022] POST /identity/v3/projects => generated 379 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:48:00.371413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:48:00.373550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:00.408787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd867ef-ca1f-4ec2-976d-0f4311aa504a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:48:00.509599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad277f3d-676e-43cc-8ea2-69a76df27582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:48:00.510245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277f3d-676e-43cc-8ea2-69a76df27582 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:00.510520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277f3d-676e-43cc-8ea2-69a76df27582 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:00.510785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad277f3d-676e-43cc-8ea2-69a76df27582 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:00.511419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad277f3d-676e-43cc-8ea2-69a76df27582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:48:00.512431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: WARNING py.warnings [None req-ad277f3d-676e-43cc-8ea2-69a76df27582 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:48:00.512431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: warnings.warn(msg) Aug 09 16:48:00.512431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: Aug 09 16:48:00.514406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad277f3d-676e-43cc-8ea2-69a76df27582 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:48:00.621598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 745/1498] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:48:00 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:48:00.627258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba548e73-e8c9-411f-ab6d-1f78e488e8c4 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:48:00.628785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ba548e73-e8c9-411f-ab6d-1f78e488e8c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:00.694021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba548e73-e8c9-411f-ab6d-1f78e488e8c4 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:48:00.779429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:00.789998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de75d5-f982-4daf-ac51-36cd7c5da575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:48:00.794947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de75d5-f982-4daf-ac51-36cd7c5da575 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:00.794947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de75d5-f982-4daf-ac51-36cd7c5da575 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:00.794947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de75d5-f982-4daf-ac51-36cd7c5da575 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:00.794947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de75d5-f982-4daf-ac51-36cd7c5da575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:48:00.794947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: WARNING py.warnings [None req-f1de75d5-f982-4daf-ac51-36cd7c5da575 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:48:00.794947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: warnings.warn(msg) Aug 09 16:48:00.794947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: Aug 09 16:48:00.795741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de75d5-f982-4daf-ac51-36cd7c5da575 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:48:00.807872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 754/1499] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:48:00.812736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad277f3d-676e-43cc-8ea2-69a76df27582 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:48:00.814430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ad277f3d-676e-43cc-8ea2-69a76df27582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:00.826292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad277f3d-676e-43cc-8ea2-69a76df27582 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:48:00.852472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:00.928261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2f9e0-5279-4cc3-9f32-c32d455d6033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:48:00.928773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2f9e0-5279-4cc3-9f32-c32d455d6033 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:00.929055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2f9e0-5279-4cc3-9f32-c32d455d6033 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:00.929322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2f9e0-5279-4cc3-9f32-c32d455d6033 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:00.929976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2f9e0-5279-4cc3-9f32-c32d455d6033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:48:00.931256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: WARNING py.warnings [None req-1ce2f9e0-5279-4cc3-9f32-c32d455d6033 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:48:00.931256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: warnings.warn(msg) Aug 09 16:48:00.931256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: Aug 09 16:48:00.932126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2f9e0-5279-4cc3-9f32-c32d455d6033 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:48:00.944569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 746/1500] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:48:00 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:48:00.948823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de75d5-f982-4daf-ac51-36cd7c5da575 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:48:00.950377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-f1de75d5-f982-4daf-ac51-36cd7c5da575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:00.984742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de75d5-f982-4daf-ac51-36cd7c5da575 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:48:01.077423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d3f80-4cca-4be1-8c16-0e93d98d88c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:48:01.078097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d3f80-4cca-4be1-8c16-0e93d98d88c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:01.078370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d3f80-4cca-4be1-8c16-0e93d98d88c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:01.078635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d3f80-4cca-4be1-8c16-0e93d98d88c3 admin admin] PATH_INFO: `/v3/projects/55e29281cef140ab83afa588f2bc93f9/users/8a0b5236b2c54f2ba1fefd82a2b5df4f/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:01.100426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d3f80-4cca-4be1-8c16-0e93d98d88c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e29281cef140ab83afa588f2bc93f9, user_id=8a0b5236b2c54f2ba1fefd82a2b5df4f, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:48:01.101442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: WARNING py.warnings [None req-9d2d3f80-4cca-4be1-8c16-0e93d98d88c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:48:01.101442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: warnings.warn(msg) Aug 09 16:48:01.101442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: Aug 09 16:48:01.103386 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d3f80-4cca-4be1-8c16-0e93d98d88c3 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:48:01.185832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 755/1501] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:48:00 2022] PUT /identity/v3/projects/55e29281cef140ab83afa588f2bc93f9/users/8a0b5236b2c54f2ba1fefd82a2b5df4f/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:48:01.198299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819fbc1-59b0-4bfd-9a58-e36d3a1fdcf2 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:01.201670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819fbc1-59b0-4bfd-9a58-e36d3a1fdcf2 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:01.201670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819fbc1-59b0-4bfd-9a58-e36d3a1fdcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:01.261682 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d819fbc1-59b0-4bfd-9a58-e36d3a1fdcf2 None None] MFA Rules not processed for user `8a0b5236b2c54f2ba1fefd82a2b5df4f`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:48:01.262926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d819fbc1-59b0-4bfd-9a58-e36d3a1fdcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:01.347030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 747/1502] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:48:01.360170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68b84-434c-469c-811b-ea7f6c5d442d None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:01.360460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68b84-434c-469c-811b-ea7f6c5d442d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:01.360733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68b84-434c-469c-811b-ea7f6c5d442d None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:01.362119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 756/1503] 10.222.0.79 () {54 vars in 964 bytes} [Tue Aug 9 16:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:48:01.367250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832c76b-589b-401c-9a84-f16bd4bf5619 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:01.367528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832c76b-589b-401c-9a84-f16bd4bf5619 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:01.367949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832c76b-589b-401c-9a84-f16bd4bf5619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:01.398842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:01.444314 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-8832c76b-589b-401c-9a84-f16bd4bf5619 None None] MFA Rules not processed for user `289e7caceab944e9b90f2e9298fb2cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:48:01.445427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8832c76b-589b-401c-9a84-f16bd4bf5619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:01.521012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 748/1504] 10.222.0.79 () {58 vars in 1061 bytes} [Tue Aug 9 16:48:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:48:01.528929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e68b84-434c-469c-811b-ea7f6c5d442d None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:48:01.531006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-16e68b84-434c-469c-811b-ea7f6c5d442d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:01.594753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:01.650814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e68b84-434c-469c-811b-ea7f6c5d442d None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:48:01.725130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208cce6-7bfa-4ba4-8c2c-169770055760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:48:01.725727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208cce6-7bfa-4ba4-8c2c-169770055760 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:01.726031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208cce6-7bfa-4ba4-8c2c-169770055760 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:01.726304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208cce6-7bfa-4ba4-8c2c-169770055760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:01.727907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b208cce6-7bfa-4ba4-8c2c-169770055760 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:01.827926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208cce6-7bfa-4ba4-8c2c-169770055760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:48:01.829360 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208cce6-7bfa-4ba4-8c2c-169770055760 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:48:01.832953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:01.847691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 757/1505] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:48:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:48:01.895242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c364e0a-d513-470e-9956-264eafe5a0d5 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:01.895697 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c364e0a-d513-470e-9956-264eafe5a0d5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:01.895821 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c364e0a-d513-470e-9956-264eafe5a0d5 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:01.897119 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 749/1506] 10.222.0.79 () {52 vars in 830 bytes} [Tue Aug 9 16:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:48:01.900967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208cce6-7bfa-4ba4-8c2c-169770055760 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:48:01.902273 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b208cce6-7bfa-4ba4-8c2c-169770055760 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:02.004548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208cce6-7bfa-4ba4-8c2c-169770055760 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:48:02.099522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:48:02.100135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:02.100413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:02.100675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] PATH_INFO: `/v3/users/8a0b5236b2c54f2ba1fefd82a2b5df4f` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:02.113462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a0b5236b2c54f2ba1fefd82a2b5df4f)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:48:02.114477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: WARNING py.warnings [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:48:02.114477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: warnings.warn(msg) Aug 09 16:48:02.114477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: Aug 09 16:48:02.115672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:48:02.126288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:02.308663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a0b5236b2c54f2ba1fefd82a2b5df4f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:48:02.388877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a0b5236b2c54f2ba1fefd82a2b5df4f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:48:02.403230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a0b5236b2c54f2ba1fefd82a2b5df4f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:48:02.427349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a0b5236b2c54f2ba1fefd82a2b5df4f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:48:02.427923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a0b5236b2c54f2ba1fefd82a2b5df4f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:48:02.433651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 758/1507] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:48:01 2022] DELETE /identity/v3/users/8a0b5236b2c54f2ba1fefd82a2b5df4f => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:48:02.438856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107cd9a-29c2-464f-a94e-93bb87e17040 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:02.439046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107cd9a-29c2-464f-a94e-93bb87e17040 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:02.439239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107cd9a-29c2-464f-a94e-93bb87e17040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:02.498103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-a107cd9a-29c2-464f-a94e-93bb87e17040 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:48:02.499322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a107cd9a-29c2-464f-a94e-93bb87e17040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:02.579130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:02.592261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 750/1508] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:48:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:48:02.602449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:48:02.603766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:02.704814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ae8172-bfc8-4647-ab76-a1a7010ad20d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:48:02.759851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:02.778306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af60f2-6e63-464c-a37f-d1de627c6359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:48:02.779505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af60f2-6e63-464c-a37f-d1de627c6359 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:02.779776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af60f2-6e63-464c-a37f-d1de627c6359 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:02.780037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af60f2-6e63-464c-a37f-d1de627c6359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:02.781634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-85af60f2-6e63-464c-a37f-d1de627c6359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:02.896719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af60f2-6e63-464c-a37f-d1de627c6359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:48:02.898082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af60f2-6e63-464c-a37f-d1de627c6359 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:48:02.917029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 759/1509] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:48:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:48:03.412243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107cd9a-29c2-464f-a94e-93bb87e17040 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:48:03.413676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a107cd9a-29c2-464f-a94e-93bb87e17040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:48:03.426300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107cd9a-29c2-464f-a94e-93bb87e17040 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:48:03.534349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:48:03.534973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:48:03.535334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:48:03.535527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] PATH_INFO: `/v3/projects/55e29281cef140ab83afa588f2bc93f9` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:48:03.546710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e29281cef140ab83afa588f2bc93f9)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:48:03.547725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: WARNING py.warnings [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:48:03.547725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: warnings.warn(msg) Aug 09 16:48:03.547725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: Aug 09 16:48:03.548905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:48:03.687285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e29281cef140ab83afa588f2bc93f9', 'request_id': 'req-7f20af4a-45f3-4255-942f-73c2e26fd740'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:48:03.748516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e29281cef140ab83afa588f2bc93f9', 'request_id': 'req-7f20af4a-45f3-4255-942f-73c2e26fd740'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:48:03.762903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e29281cef140ab83afa588f2bc93f9', 'request_id': 'req-7f20af4a-45f3-4255-942f-73c2e26fd740'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:48:03.786035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] The token cache is being invalidate because project 55e29281cef140ab83afa588f2bc93f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:48:03.786330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:48:03.786863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7f20af4a-45f3-4255-942f-73c2e26fd740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:48:03.788175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 751/1510] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:48:03 2022] DELETE /identity/v3/projects/55e29281cef140ab83afa588f2bc93f9 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:48:12.914765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=62 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:48:13.783875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:49:37.359875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd9860-38d1-43f7-9a87-418c954d268f None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:37.360487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd9860-38d1-43f7-9a87-418c954d268f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:37.360487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd9860-38d1-43f7-9a87-418c954d268f None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:37.361863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:49:37.367561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23235048-11dd-4263-a56c-7b2e2e00a41f None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:37.368024 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23235048-11dd-4263-a56c-7b2e2e00a41f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:37.368141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23235048-11dd-4263-a56c-7b2e2e00a41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:37.456509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-23235048-11dd-4263-a56c-7b2e2e00a41f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:49:37.457850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-23235048-11dd-4263-a56c-7b2e2e00a41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:37.565357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:49:37.719273 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd9860-38d1-43f7-9a87-418c954d268f None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:49:37.721534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0ebd9860-38d1-43f7-9a87-418c954d268f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:37.841418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd9860-38d1-43f7-9a87-418c954d268f None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:49:37.847438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed63d6c-090f-40af-9dce-1a445ef63ad4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:49:37.848030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed63d6c-090f-40af-9dce-1a445ef63ad4 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:37.848316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed63d6c-090f-40af-9dce-1a445ef63ad4 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:37.848589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed63d6c-090f-40af-9dce-1a445ef63ad4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:37.856912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed63d6c-090f-40af-9dce-1a445ef63ad4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:49:37.858522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed63d6c-090f-40af-9dce-1a445ef63ad4 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:49:37.868468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 761/1513] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:49:38.705787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccdc59-75d8-4c68-ac00-bd486fa0c3a5 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:38.706405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccdc59-75d8-4c68-ac00-bd486fa0c3a5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:38.706405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccdc59-75d8-4c68-ac00-bd486fa0c3a5 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:38.707689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:49:38.713256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbee15-3a09-48dd-acac-47172ecfbf5f None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:38.713735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbee15-3a09-48dd-acac-47172ecfbf5f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:38.713838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbee15-3a09-48dd-acac-47172ecfbf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:38.720150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66e626-3418-4b59-8c51-be4031529b61 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:38.720549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66e626-3418-4b59-8c51-be4031529b61 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:38.720911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66e626-3418-4b59-8c51-be4031529b61 None None] PATH_INFO: `/v3` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:38.722627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 754/1515] 10.222.0.100 () {54 vars in 926 bytes} [Tue Aug 9 16:49:38 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:49:38.731216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd2520-01c2-44d6-b044-20762864274b None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:38.731496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd2520-01c2-44d6-b044-20762864274b None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:38.731765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd2520-01c2-44d6-b044-20762864274b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:38.736303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-3edbee15-3a09-48dd-acac-47172ecfbf5f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:49:38.737517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3edbee15-3a09-48dd-acac-47172ecfbf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:38.746411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 762/1516] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:49:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:49:38.803104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2ddd2520-01c2-44d6-b044-20762864274b None None] MFA Rules not processed for user `d7bc938254514471801c88761525e883`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:49:38.804254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2ddd2520-01c2-44d6-b044-20762864274b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:38.836549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:49:38.886273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 755/1517] 10.222.0.100 () {58 vars in 1016 bytes} [Tue Aug 9 16:49:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:49:38.900907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbee15-3a09-48dd-acac-47172ecfbf5f None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:49:38.913121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbee15-3a09-48dd-acac-47172ecfbf5f None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:49:38.918648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2d24a-5e25-4f04-a240-bef9941fcf53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:49:38.919238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2d24a-5e25-4f04-a240-bef9941fcf53 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:38.919513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2d24a-5e25-4f04-a240-bef9941fcf53 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:38.919781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2d24a-5e25-4f04-a240-bef9941fcf53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:38.928609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2d24a-5e25-4f04-a240-bef9941fcf53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:49:38.930191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2d24a-5e25-4f04-a240-bef9941fcf53 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:49:38.939991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 763/1518] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:49:48.159214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f634a2b-a519-4d10-8ae1-f735f8ebf2fa None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:48.159784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f634a2b-a519-4d10-8ae1-f735f8ebf2fa None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:48.159784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f634a2b-a519-4d10-8ae1-f735f8ebf2fa None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:48.160981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 756/1519] 10.222.0.79 () {54 vars in 961 bytes} [Tue Aug 9 16:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:49:48.165997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cd9b7-4333-46ac-8cbd-8a37d7440963 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:48.166465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cd9b7-4333-46ac-8cbd-8a37d7440963 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:48.166579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cd9b7-4333-46ac-8cbd-8a37d7440963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:48.185921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-790cd9b7-4333-46ac-8cbd-8a37d7440963 None None] MFA Rules not processed for user `d7bc938254514471801c88761525e883`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:49:48.187139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-790cd9b7-4333-46ac-8cbd-8a37d7440963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:48.196076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 764/1520] 10.222.0.79 () {58 vars in 1058 bytes} [Tue Aug 9 16:49:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:49:48.203585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f634a2b-a519-4d10-8ae1-f735f8ebf2fa None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:49:48.216735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f634a2b-a519-4d10-8ae1-f735f8ebf2fa None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:49:48.221524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6e1a9-6fc9-4118-864a-eea3d270a695 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:49:48.222138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6e1a9-6fc9-4118-864a-eea3d270a695 service placement] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:48.222413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6e1a9-6fc9-4118-864a-eea3d270a695 service placement] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:48.222681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6e1a9-6fc9-4118-864a-eea3d270a695 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:48.230871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6e1a9-6fc9-4118-864a-eea3d270a695 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:49:48.232431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6e1a9-6fc9-4118-864a-eea3d270a695 service placement] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:49:48.242599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 757/1521] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:49:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:49:50.158260 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e75a3-5d61-4d0e-976c-15d4051dc734 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:50.158915 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e75a3-5d61-4d0e-976c-15d4051dc734 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:50.158915 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e75a3-5d61-4d0e-976c-15d4051dc734 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:50.160155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:49:50.164901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64128-5ffb-4aa4-9c18-aab86ae9ab5d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:50.165343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64128-5ffb-4aa4-9c18-aab86ae9ab5d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:50.165483 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64128-5ffb-4aa4-9c18-aab86ae9ab5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:50.185437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2ac64128-5ffb-4aa4-9c18-aab86ae9ab5d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:49:50.186644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2ac64128-5ffb-4aa4-9c18-aab86ae9ab5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:50.194410 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:49:50.347922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e75a3-5d61-4d0e-976c-15d4051dc734 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:49:50.363023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e75a3-5d61-4d0e-976c-15d4051dc734 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:49:50.368573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d2a0c-5054-436b-a231-97c558f05839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:49:50.369159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d2a0c-5054-436b-a231-97c558f05839 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:50.369433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d2a0c-5054-436b-a231-97c558f05839 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:50.369836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d2a0c-5054-436b-a231-97c558f05839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:50.378173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d2a0c-5054-436b-a231-97c558f05839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:49:50.379845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d2a0c-5054-436b-a231-97c558f05839 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:49:50.389356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 766/1524] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:49:51.219165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274e81b-49a2-4934-8bdd-b86eb608763c None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:51.219746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274e81b-49a2-4934-8bdd-b86eb608763c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:51.219746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274e81b-49a2-4934-8bdd-b86eb608763c None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:51.221631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 9 16:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:49:51.226476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9836b9b-3004-4de0-a567-18f1c89f52d0 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:51.226916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9836b9b-3004-4de0-a567-18f1c89f52d0 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:51.227023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9836b9b-3004-4de0-a567-18f1c89f52d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:51.245965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-b9836b9b-3004-4de0-a567-18f1c89f52d0 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:49:51.247187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b9836b9b-3004-4de0-a567-18f1c89f52d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:51.254948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 9 16:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:49:51.406728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274e81b-49a2-4934-8bdd-b86eb608763c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:49:51.421345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274e81b-49a2-4934-8bdd-b86eb608763c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:49:51.427413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222cb8c-a2d5-49a3-8390-d9427cfdfd0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:49:51.442449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222cb8c-a2d5-49a3-8390-d9427cfdfd0f service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:51.442449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222cb8c-a2d5-49a3-8390-d9427cfdfd0f service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:51.442449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222cb8c-a2d5-49a3-8390-d9427cfdfd0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:51.442449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222cb8c-a2d5-49a3-8390-d9427cfdfd0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:49:51.442449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222cb8c-a2d5-49a3-8390-d9427cfdfd0f service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:49:51.449508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 760/1527] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:49:55.743595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34895fdb-5de8-430e-bb5f-7eb8f613c8e5 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:55.744627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34895fdb-5de8-430e-bb5f-7eb8f613c8e5 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:55.744627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34895fdb-5de8-430e-bb5f-7eb8f613c8e5 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:55.745469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:49:55.750798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc594c17-7d10-45dd-988d-3465a6f07975 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:55.751241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc594c17-7d10-45dd-988d-3465a6f07975 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:55.751350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc594c17-7d10-45dd-988d-3465a6f07975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:55.771240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-cc594c17-7d10-45dd-988d-3465a6f07975 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:49:55.779231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-cc594c17-7d10-45dd-988d-3465a6f07975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:55.781491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:49:55.855250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34895fdb-5de8-430e-bb5f-7eb8f613c8e5 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:49:55.856943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-34895fdb-5de8-430e-bb5f-7eb8f613c8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:55.920999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:49:55.936407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34895fdb-5de8-430e-bb5f-7eb8f613c8e5 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:49:55.940223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b94def-837f-49e4-9816-a220432a03b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:49:55.940816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b94def-837f-49e4-9816-a220432a03b1 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:55.941095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b94def-837f-49e4-9816-a220432a03b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:55.941367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b94def-837f-49e4-9816-a220432a03b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:55.950913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b94def-837f-49e4-9816-a220432a03b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:49:55.952747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b94def-837f-49e4-9816-a220432a03b1 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:49:55.962409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 769/1530] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:49:57.954766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1c2bb-0018-4d24-bc07-374980d181be None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:57.955787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1c2bb-0018-4d24-bc07-374980d181be None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:57.955787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1c2bb-0018-4d24-bc07-374980d181be None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:57.957052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:49:57.962321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405dbda-6b6d-4e61-8c2b-19101f603d4a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:57.962757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405dbda-6b6d-4e61-8c2b-19101f603d4a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:57.962872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405dbda-6b6d-4e61-8c2b-19101f603d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:57.982685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-e405dbda-6b6d-4e61-8c2b-19101f603d4a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:49:57.983885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e405dbda-6b6d-4e61-8c2b-19101f603d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:57.991727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:49:58.032175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1c2bb-0018-4d24-bc07-374980d181be None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:49:58.034633 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a6e1c2bb-0018-4d24-bc07-374980d181be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:49:58.047704 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1c2bb-0018-4d24-bc07-374980d181be None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:49:58.051437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ae32e-206c-48d4-8945-1ae284000789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:49:58.052030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ae32e-206c-48d4-8945-1ae284000789 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:49:58.052307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ae32e-206c-48d4-8945-1ae284000789 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:49:58.052574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ae32e-206c-48d4-8945-1ae284000789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:49:58.061941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ae32e-206c-48d4-8945-1ae284000789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:49:58.063390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ae32e-206c-48d4-8945-1ae284000789 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:49:58.074098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 763/1533] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:50:01.616714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7021375-9228-47e0-984a-6b980b6783c8 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:50:01.617293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7021375-9228-47e0-984a-6b980b6783c8 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:50:01.617293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7021375-9228-47e0-984a-6b980b6783c8 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:50:01.618602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:50:01.623755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005983a5-bd32-4b02-a5d6-2cc68f9f4a0f None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:50:01.624216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005983a5-bd32-4b02-a5d6-2cc68f9f4a0f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:50:01.624324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005983a5-bd32-4b02-a5d6-2cc68f9f4a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:50:01.643913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-005983a5-bd32-4b02-a5d6-2cc68f9f4a0f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:50:01.645108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-005983a5-bd32-4b02-a5d6-2cc68f9f4a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:50:01.652942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:50:01.954097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7021375-9228-47e0-984a-6b980b6783c8 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:50:01.956032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a7021375-9228-47e0-984a-6b980b6783c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:50:02.032588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7021375-9228-47e0-984a-6b980b6783c8 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:50:02.036692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9f757-01b1-4e5e-938c-5e9c46a85955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:50:02.037288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9f757-01b1-4e5e-938c-5e9c46a85955 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:50:02.037566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9f757-01b1-4e5e-938c-5e9c46a85955 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:50:02.037837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9f757-01b1-4e5e-938c-5e9c46a85955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:50:02.046759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9f757-01b1-4e5e-938c-5e9c46a85955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:50:02.048449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9f757-01b1-4e5e-938c-5e9c46a85955 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:50:02.058339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 772/1536] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:50:04.080760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9886649-c187-450c-a10a-e0990fe4fd08 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:50:04.081314 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9886649-c187-450c-a10a-e0990fe4fd08 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:50:04.081314 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9886649-c187-450c-a10a-e0990fe4fd08 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:50:04.082674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 9 16:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:50:04.087936 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ccf61-6e09-4c1c-8b25-f2dd177a1444 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:50:04.088415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ccf61-6e09-4c1c-8b25-f2dd177a1444 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:50:04.088521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ccf61-6e09-4c1c-8b25-f2dd177a1444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:50:04.107991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a86ccf61-6e09-4c1c-8b25-f2dd177a1444 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:50:04.109249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a86ccf61-6e09-4c1c-8b25-f2dd177a1444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:50:04.117604 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 9 16:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:50:04.425578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9886649-c187-450c-a10a-e0990fe4fd08 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:50:04.427381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e9886649-c187-450c-a10a-e0990fe4fd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:50:04.440652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9886649-c187-450c-a10a-e0990fe4fd08 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:50:04.444260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd465da7-de3e-418c-a9fa-50a657bc6c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:50:04.444903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd465da7-de3e-418c-a9fa-50a657bc6c04 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:50:04.445179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd465da7-de3e-418c-a9fa-50a657bc6c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:50:04.445448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd465da7-de3e-418c-a9fa-50a657bc6c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:50:04.454506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd465da7-de3e-418c-a9fa-50a657bc6c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:50:04.456456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd465da7-de3e-418c-a9fa-50a657bc6c04 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:50:04.465911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 766/1539] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:50:14.103111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=50 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:50:14.461780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=50 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:31.510040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642996f3-59be-4285-8fc9-887bfaa05b38 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:31.556149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642996f3-59be-4285-8fc9-887bfaa05b38 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:31.556149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642996f3-59be-4285-8fc9-887bfaa05b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:31.556149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-642996f3-59be-4285-8fc9-887bfaa05b38 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:31.556149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-642996f3-59be-4285-8fc9-887bfaa05b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:31.556149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 774/1540] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:31.565150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eceb32-750e-4d35-9330-8b9a3603f133 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:31.565150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eceb32-750e-4d35-9330-8b9a3603f133 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:31.565150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eceb32-750e-4d35-9330-8b9a3603f133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:31.589851 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f8eceb32-750e-4d35-9330-8b9a3603f133 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:31.591468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f8eceb32-750e-4d35-9330-8b9a3603f133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:31.603374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 767/1541] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:31.607701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642996f3-59be-4285-8fc9-887bfaa05b38 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:31.623664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642996f3-59be-4285-8fc9-887bfaa05b38 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:31.634837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd34e78-0d62-48ed-9bb2-4791d10d398f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:31.635453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd34e78-0d62-48ed-9bb2-4791d10d398f admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:31.635725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd34e78-0d62-48ed-9bb2-4791d10d398f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:31.635987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd34e78-0d62-48ed-9bb2-4791d10d398f admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:31.636642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd34e78-0d62-48ed-9bb2-4791d10d398f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:51:31.636955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd34e78-0d62-48ed-9bb2-4791d10d398f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:31.637991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: WARNING py.warnings [None req-8bd34e78-0d62-48ed-9bb2-4791d10d398f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:51:31.637991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: warnings.warn(msg) Aug 09 16:51:31.637991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: Aug 09 16:51:31.638894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd34e78-0d62-48ed-9bb2-4791d10d398f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:31.661700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 775/1542] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:51:31.669569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eceb32-750e-4d35-9330-8b9a3603f133 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:31.680280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eceb32-750e-4d35-9330-8b9a3603f133 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:31.688584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5d7af-4890-49db-9bec-d384501e6e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:31.689510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5d7af-4890-49db-9bec-d384501e6e88 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:31.690040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5d7af-4890-49db-9bec-d384501e6e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:31.690544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5d7af-4890-49db-9bec-d384501e6e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:31.691384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5d7af-4890-49db-9bec-d384501e6e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:31.692615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: WARNING py.warnings [None req-03b5d7af-4890-49db-9bec-d384501e6e88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:51:31.692615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: warnings.warn(msg) Aug 09 16:51:31.692615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: Aug 09 16:51:31.694225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5d7af-4890-49db-9bec-d384501e6e88 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:31.775156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdc028-ab3e-4b88-8525-6f1cdaa50a0c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:31.775359 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdc028-ab3e-4b88-8525-6f1cdaa50a0c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:31.775602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fdc028-ab3e-4b88-8525-6f1cdaa50a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:31.804495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-48fdc028-ab3e-4b88-8525-6f1cdaa50a0c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:31.805709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-48fdc028-ab3e-4b88-8525-6f1cdaa50a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:31.818069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 776/1543] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:51:31.823506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 768/1544] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:31 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:31.827131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafaa57-35d6-480a-a99b-acf36b96ae3b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:31.827561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafaa57-35d6-480a-a99b-acf36b96ae3b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:31.827836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafaa57-35d6-480a-a99b-acf36b96ae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:31.829058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5d7af-4890-49db-9bec-d384501e6e88 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:31.861344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9aafaa57-35d6-480a-a99b-acf36b96ae3b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:31.861697 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9aafaa57-35d6-480a-a99b-acf36b96ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:31.948598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5d7af-4890-49db-9bec-d384501e6e88 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:31.966772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b4d81-5a33-494c-add2-8b5f73bd646f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:31.967640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b4d81-5a33-494c-add2-8b5f73bd646f admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:31.967640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b4d81-5a33-494c-add2-8b5f73bd646f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:31.967930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b4d81-5a33-494c-add2-8b5f73bd646f admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:31.968652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b4d81-5a33-494c-add2-8b5f73bd646f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:31.971012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b4d81-5a33-494c-add2-8b5f73bd646f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.001105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 777/1545] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.002783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859bf68-5f53-40f6-ba24-7eb9d19b09a6 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.003078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859bf68-5f53-40f6-ba24-7eb9d19b09a6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.003349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859bf68-5f53-40f6-ba24-7eb9d19b09a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.036921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d859bf68-5f53-40f6-ba24-7eb9d19b09a6 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.038113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d859bf68-5f53-40f6-ba24-7eb9d19b09a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.055939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 778/1546] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.062247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859bf68-5f53-40f6-ba24-7eb9d19b09a6 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.070112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859bf68-5f53-40f6-ba24-7eb9d19b09a6 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.080986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2a718-2d63-405e-8000-76aec04f3f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.081623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2a718-2d63-405e-8000-76aec04f3f4b admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.081901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2a718-2d63-405e-8000-76aec04f3f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.082195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2a718-2d63-405e-8000-76aec04f3f4b admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.082867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2a718-2d63-405e-8000-76aec04f3f4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:51:32.083168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2a718-2d63-405e-8000-76aec04f3f4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.085005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2a718-2d63-405e-8000-76aec04f3f4b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.098707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 769/1547] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:31 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:51:32.100054 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129edc2-691a-4a30-8ef7-407c16d1b8f7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.100330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129edc2-691a-4a30-8ef7-407c16d1b8f7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.100597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129edc2-691a-4a30-8ef7-407c16d1b8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.101807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 779/1548] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:51:32.103162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed861578-d00b-4975-b727-f6644e109dee None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.103445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed861578-d00b-4975-b727-f6644e109dee None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.103768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed861578-d00b-4975-b727-f6644e109dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.136711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ed861578-d00b-4975-b727-f6644e109dee None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.137944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ed861578-d00b-4975-b727-f6644e109dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.143529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b129edc2-691a-4a30-8ef7-407c16d1b8f7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.145025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b129edc2-691a-4a30-8ef7-407c16d1b8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.157069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 780/1549] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.159657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed861578-d00b-4975-b727-f6644e109dee None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.161351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 770/1550] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.166322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5726e-6fe4-492c-9ade-25fdb4b759f1 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.166850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5726e-6fe4-492c-9ade-25fdb4b759f1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.167116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5726e-6fe4-492c-9ade-25fdb4b759f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.177206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed861578-d00b-4975-b727-f6644e109dee None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.198037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb8cb0-b75f-4070-a380-b0e2661933d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.198651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb8cb0-b75f-4070-a380-b0e2661933d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.199040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb8cb0-b75f-4070-a380-b0e2661933d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.199314 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb8cb0-b75f-4070-a380-b0e2661933d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.199955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb8cb0-b75f-4070-a380-b0e2661933d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.201549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb8cb0-b75f-4070-a380-b0e2661933d8 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.216005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-16f5726e-6fe4-492c-9ade-25fdb4b759f1 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.217202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-16f5726e-6fe4-492c-9ade-25fdb4b759f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.224892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 781/1551] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:32.228062 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb8cb0-b75f-4070-a380-b0e2661933d8 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.238511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 771/1552] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.239559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5726e-6fe4-492c-9ade-25fdb4b759f1 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.247342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb8cb0-b75f-4070-a380-b0e2661933d8 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.253593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93617926-a275-4e93-9952-cbb1affc43b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.254550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5726e-6fe4-492c-9ade-25fdb4b759f1 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.254696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617926-a275-4e93-9952-cbb1affc43b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.254917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617926-a275-4e93-9952-cbb1affc43b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.255191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617926-a275-4e93-9952-cbb1affc43b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.255825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93617926-a275-4e93-9952-cbb1affc43b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.257725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93617926-a275-4e93-9952-cbb1affc43b1 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.259086 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e17d05-d0fb-4507-b2a9-0372776fe709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.259684 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e17d05-d0fb-4507-b2a9-0372776fe709 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.260084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e17d05-d0fb-4507-b2a9-0372776fe709 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.260448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e17d05-d0fb-4507-b2a9-0372776fe709 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.261180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e17d05-d0fb-4507-b2a9-0372776fe709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:51:32.261476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e17d05-d0fb-4507-b2a9-0372776fe709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.263052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e17d05-d0fb-4507-b2a9-0372776fe709 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.276574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 772/1553] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:51:32.277900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c6fe-598b-47e2-b3fb-5bf765187512 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.278210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c6fe-598b-47e2-b3fb-5bf765187512 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.278481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c6fe-598b-47e2-b3fb-5bf765187512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.302631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c338c6fe-598b-47e2-b3fb-5bf765187512 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.303751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c338c6fe-598b-47e2-b3fb-5bf765187512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.311646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 773/1554] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.312429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c6fe-598b-47e2-b3fb-5bf765187512 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.323361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c6fe-598b-47e2-b3fb-5bf765187512 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.327928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a189a0-102c-47fd-9c5c-c932d1e2d1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.328574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a189a0-102c-47fd-9c5c-c932d1e2d1fd admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.328850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a189a0-102c-47fd-9c5c-c932d1e2d1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.334799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a189a0-102c-47fd-9c5c-c932d1e2d1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.334799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a189a0-102c-47fd-9c5c-c932d1e2d1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.334799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a189a0-102c-47fd-9c5c-c932d1e2d1fd admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.342741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 774/1555] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:32.343936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6bcb2-518d-4529-853c-cfba345e27ce None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.344216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6bcb2-518d-4529-853c-cfba345e27ce None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.344486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6bcb2-518d-4529-853c-cfba345e27ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.359264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 782/1556] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:32.359452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b68d5-0dcb-4ab3-b0a2-41ec67726e82 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.362756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b68d5-0dcb-4ab3-b0a2-41ec67726e82 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.362756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b68d5-0dcb-4ab3-b0a2-41ec67726e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.364041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c3d6bcb2-518d-4529-853c-cfba345e27ce None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.365218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c3d6bcb2-518d-4529-853c-cfba345e27ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.379334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-bf5b68d5-0dcb-4ab3-b0a2-41ec67726e82 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.380513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bf5b68d5-0dcb-4ab3-b0a2-41ec67726e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.424945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 775/1557] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.426510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fd21a-385d-4dbf-870d-384151578516 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.426779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fd21a-385d-4dbf-870d-384151578516 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.427322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4fd21a-385d-4dbf-870d-384151578516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.439821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 783/1558] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.442656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b68d5-0dcb-4ab3-b0a2-41ec67726e82 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.446919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-4e4fd21a-385d-4dbf-870d-384151578516 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.448548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4e4fd21a-385d-4dbf-870d-384151578516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.452001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b68d5-0dcb-4ab3-b0a2-41ec67726e82 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.456994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc066f2-19f1-49aa-9a5f-83464a587255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.463372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 776/1559] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.463516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc066f2-19f1-49aa-9a5f-83464a587255 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.463516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc066f2-19f1-49aa-9a5f-83464a587255 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.463516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc066f2-19f1-49aa-9a5f-83464a587255 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.463767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fd21a-385d-4dbf-870d-384151578516 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.463845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc066f2-19f1-49aa-9a5f-83464a587255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.463845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc066f2-19f1-49aa-9a5f-83464a587255 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.463975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:32.469594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4fd21a-385d-4dbf-870d-384151578516 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.474587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f8ce8-0214-48c3-ab70-6a0125f2b8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.475204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8ce8-0214-48c3-ab70-6a0125f2b8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.475479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8ce8-0214-48c3-ab70-6a0125f2b8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.475745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f8ce8-0214-48c3-ab70-6a0125f2b8f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.476409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f8ce8-0214-48c3-ab70-6a0125f2b8f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:51:32.476703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f8ce8-0214-48c3-ab70-6a0125f2b8f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.477154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:32.478313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f8ce8-0214-48c3-ab70-6a0125f2b8f4 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.489870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 777/1560] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:51:32.490939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc05109-694f-46ad-b279-333bdcf694c7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.491223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc05109-694f-46ad-b279-333bdcf694c7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.491536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc05109-694f-46ad-b279-333bdcf694c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.511302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2bc05109-694f-46ad-b279-333bdcf694c7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.512518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2bc05109-694f-46ad-b279-333bdcf694c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.521740 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 778/1561] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.522696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc05109-694f-46ad-b279-333bdcf694c7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.535385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc05109-694f-46ad-b279-333bdcf694c7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.536892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bedb50-3f83-43ff-b7b2-6130ec36e613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.537533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bedb50-3f83-43ff-b7b2-6130ec36e613 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.537869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bedb50-3f83-43ff-b7b2-6130ec36e613 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.538320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bedb50-3f83-43ff-b7b2-6130ec36e613 admin admin] PATH_INFO: `/v3/projects/8fa9b79fe82243618fb102355a1c8ec5/users/071cf79ebf6245afa2b7f0d23f9393e4/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.553081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bedb50-3f83-43ff-b7b2-6130ec36e613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa9b79fe82243618fb102355a1c8ec5, user_id=071cf79ebf6245afa2b7f0d23f9393e4, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.554200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: WARNING py.warnings [None req-b7bedb50-3f83-43ff-b7b2-6130ec36e613 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:51:32.554200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: warnings.warn(msg) Aug 09 16:51:32.554200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: Aug 09 16:51:32.556123 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bedb50-3f83-43ff-b7b2-6130ec36e613 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.557875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 784/1562] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:32.562190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc066f2-19f1-49aa-9a5f-83464a587255 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.659615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 779/1563] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:32 2022] PUT /identity/v3/projects/8fa9b79fe82243618fb102355a1c8ec5/users/071cf79ebf6245afa2b7f0d23f9393e4/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:32.660477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bedb50-3f83-43ff-b7b2-6130ec36e613 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.714999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc066f2-19f1-49aa-9a5f-83464a587255 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.721157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47067e14-e4cb-4b38-b010-7090d1bacdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.721761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47067e14-e4cb-4b38-b010-7090d1bacdb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.722117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47067e14-e4cb-4b38-b010-7090d1bacdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.722393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47067e14-e4cb-4b38-b010-7090d1bacdb0 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.723020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47067e14-e4cb-4b38-b010-7090d1bacdb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.724033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: WARNING py.warnings [None req-47067e14-e4cb-4b38-b010-7090d1bacdb0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:51:32.724033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: warnings.warn(msg) Aug 09 16:51:32.724033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: Aug 09 16:51:32.724291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bedb50-3f83-43ff-b7b2-6130ec36e613 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.725230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47067e14-e4cb-4b38-b010-7090d1bacdb0 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.728831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5bff6-7d96-4658-96f6-60acf5f99b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.729431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5bff6-7d96-4658-96f6-60acf5f99b42 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.729707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5bff6-7d96-4658-96f6-60acf5f99b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.730008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5bff6-7d96-4658-96f6-60acf5f99b42 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.730651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5bff6-7d96-4658-96f6-60acf5f99b42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:51:32.731036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5bff6-7d96-4658-96f6-60acf5f99b42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.732599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5bff6-7d96-4658-96f6-60acf5f99b42 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.745437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 780/1564] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:51:32.746660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc58fc4-c715-49d0-bc8f-69b099c3c1a9 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.746940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc58fc4-c715-49d0-bc8f-69b099c3c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.747216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc58fc4-c715-49d0-bc8f-69b099c3c1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.766755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-bdc58fc4-c715-49d0-bc8f-69b099c3c1a9 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.767942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bdc58fc4-c715-49d0-bc8f-69b099c3c1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.775931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 781/1565] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.777056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2dfef-6e2c-4d6f-97e0-204758f23e71 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.777376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2dfef-6e2c-4d6f-97e0-204758f23e71 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.777647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2dfef-6e2c-4d6f-97e0-204758f23e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.797302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c6b2dfef-6e2c-4d6f-97e0-204758f23e71 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.801039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c6b2dfef-6e2c-4d6f-97e0-204758f23e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.807045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 782/1566] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.807808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2dfef-6e2c-4d6f-97e0-204758f23e71 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.817572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2dfef-6e2c-4d6f-97e0-204758f23e71 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.822321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900239d-b9c8-4d8e-b74f-bc4843f497f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.822892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900239d-b9c8-4d8e-b74f-bc4843f497f7 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.823167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900239d-b9c8-4d8e-b74f-bc4843f497f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.823433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900239d-b9c8-4d8e-b74f-bc4843f497f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.824035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900239d-b9c8-4d8e-b74f-bc4843f497f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.825391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 785/1567] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/users => generated 294 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:32.825978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900239d-b9c8-4d8e-b74f-bc4843f497f7 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.826795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a7836-9f88-4763-a884-db81b2e6893a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.827076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a7836-9f88-4763-a884-db81b2e6893a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.829886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a7836-9f88-4763-a884-db81b2e6893a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.850231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-485a7836-9f88-4763-a884-db81b2e6893a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.851486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-485a7836-9f88-4763-a884-db81b2e6893a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.859724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 786/1568] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:32.861072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a7836-9f88-4763-a884-db81b2e6893a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.870215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a7836-9f88-4763-a884-db81b2e6893a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:32.874810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85657df4-a18c-4e7a-984b-e4142824d838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:32.875367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85657df4-a18c-4e7a-984b-e4142824d838 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.875693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85657df4-a18c-4e7a-984b-e4142824d838 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.875990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85657df4-a18c-4e7a-984b-e4142824d838 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.877150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85657df4-a18c-4e7a-984b-e4142824d838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:32.879170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85657df4-a18c-4e7a-984b-e4142824d838 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:32.906650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 783/1569] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:32.908204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2301a-1969-4f7a-b7d6-319dd662a7ff None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:32.908546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2301a-1969-4f7a-b7d6-319dd662a7ff None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:32.908925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2301a-1969-4f7a-b7d6-319dd662a7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:32.936719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b2a2301a-1969-4f7a-b7d6-319dd662a7ff None None] MFA Rules not processed for user `071cf79ebf6245afa2b7f0d23f9393e4`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:32.937940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b2a2301a-1969-4f7a-b7d6-319dd662a7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:32.959159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 787/1570] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:32.959159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85657df4-a18c-4e7a-984b-e4142824d838 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:32.966506 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85657df4-a18c-4e7a-984b-e4142824d838 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.006337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 784/1571] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:33.007179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2301a-1969-4f7a-b7d6-319dd662a7ff None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.015075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:33.026958 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662caea-65c9-4045-8586-d20ac44eff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.027664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662caea-65c9-4045-8586-d20ac44eff23 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.027932 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662caea-65c9-4045-8586-d20ac44eff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.028210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662caea-65c9-4045-8586-d20ac44eff23 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.028833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662caea-65c9-4045-8586-d20ac44eff23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.030997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662caea-65c9-4045-8586-d20ac44eff23 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.070252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2301a-1969-4f7a-b7d6-319dd662a7ff None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.074839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c09d4b-7e33-495c-9e7d-4d3a81653a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.075437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c09d4b-7e33-495c-9e7d-4d3a81653a2d admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.075787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c09d4b-7e33-495c-9e7d-4d3a81653a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.076058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c09d4b-7e33-495c-9e7d-4d3a81653a2d admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.076717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c09d4b-7e33-495c-9e7d-4d3a81653a2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:51:33.077001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c09d4b-7e33-495c-9e7d-4d3a81653a2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.078710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c09d4b-7e33-495c-9e7d-4d3a81653a2d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.090089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 785/1572] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:51:33.090871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c09d4b-7e33-495c-9e7d-4d3a81653a2d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.100339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c09d4b-7e33-495c-9e7d-4d3a81653a2d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.101426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 788/1573] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:32 2022] POST /identity/v3/projects => generated 391 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:33.102239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662caea-65c9-4045-8586-d20ac44eff23 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.148063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:33.158578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76c556-8e89-46c4-8dfc-d51ae1875d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.162300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76c556-8e89-46c4-8dfc-d51ae1875d07 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.162300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76c556-8e89-46c4-8dfc-d51ae1875d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.162300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76c556-8e89-46c4-8dfc-d51ae1875d07 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.162300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76c556-8e89-46c4-8dfc-d51ae1875d07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:51:33.162300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76c556-8e89-46c4-8dfc-d51ae1875d07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.162300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76c556-8e89-46c4-8dfc-d51ae1875d07 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.164712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662caea-65c9-4045-8586-d20ac44eff23 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.170031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb3399-7511-4355-b77b-9803fbf9b84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.170458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb3399-7511-4355-b77b-9803fbf9b84e admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.170731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb3399-7511-4355-b77b-9803fbf9b84e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.170993 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb3399-7511-4355-b77b-9803fbf9b84e admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.171704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb3399-7511-4355-b77b-9803fbf9b84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.173913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb3399-7511-4355-b77b-9803fbf9b84e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.175530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 786/1574] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:51:33.187901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76c556-8e89-46c4-8dfc-d51ae1875d07 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.187901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76c556-8e89-46c4-8dfc-d51ae1875d07 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.191231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 789/1575] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:33.191479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d07867-86f4-4fdc-ae7c-764b71798955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.192044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07867-86f4-4fdc-ae7c-764b71798955 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.192143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb3399-7511-4355-b77b-9803fbf9b84e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.192317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07867-86f4-4fdc-ae7c-764b71798955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.192589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07867-86f4-4fdc-ae7c-764b71798955 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.193222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d07867-86f4-4fdc-ae7c-764b71798955 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:51:33.193515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d07867-86f4-4fdc-ae7c-764b71798955 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.195120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d07867-86f4-4fdc-ae7c-764b71798955 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.201785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb3399-7511-4355-b77b-9803fbf9b84e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.206504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681345d-47f2-4af2-9da8-7eab6b57a3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.206968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 787/1576] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:51:33.207830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d07867-86f4-4fdc-ae7c-764b71798955 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.215173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681345d-47f2-4af2-9da8-7eab6b57a3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.215643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681345d-47f2-4af2-9da8-7eab6b57a3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.216166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681345d-47f2-4af2-9da8-7eab6b57a3c1 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.217097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681345d-47f2-4af2-9da8-7eab6b57a3c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.217855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d07867-86f4-4fdc-ae7c-764b71798955 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.219109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681345d-47f2-4af2-9da8-7eab6b57a3c1 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.223607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc3e03-6492-4ebe-91de-4a941291dd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.224211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc3e03-6492-4ebe-91de-4a941291dd26 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.224490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc3e03-6492-4ebe-91de-4a941291dd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.224760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc3e03-6492-4ebe-91de-4a941291dd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.225421 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc3e03-6492-4ebe-91de-4a941291dd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.227041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc3e03-6492-4ebe-91de-4a941291dd26 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.238058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 788/1577] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:33.238877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc3e03-6492-4ebe-91de-4a941291dd26 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.248342 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc3e03-6492-4ebe-91de-4a941291dd26 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.253528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb65864-224c-4fe3-9daf-6de0a5fc0727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.254142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb65864-224c-4fe3-9daf-6de0a5fc0727 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.254419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb65864-224c-4fe3-9daf-6de0a5fc0727 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.254691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb65864-224c-4fe3-9daf-6de0a5fc0727 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.255299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb65864-224c-4fe3-9daf-6de0a5fc0727 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.257237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb65864-224c-4fe3-9daf-6de0a5fc0727 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.316072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 790/1578] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:33 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:33.317044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681345d-47f2-4af2-9da8-7eab6b57a3c1 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.326741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681345d-47f2-4af2-9da8-7eab6b57a3c1 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.331366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f27b71-dbc2-4850-be51-2bf137c62ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.331935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f27b71-dbc2-4850-be51-2bf137c62ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.332211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f27b71-dbc2-4850-be51-2bf137c62ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.332478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f27b71-dbc2-4850-be51-2bf137c62ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.333080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f27b71-dbc2-4850-be51-2bf137c62ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.335012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f27b71-dbc2-4850-be51-2bf137c62ab2 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.346552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 789/1579] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:33 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:33.347354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb65864-224c-4fe3-9daf-6de0a5fc0727 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.397141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:33.410650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb65864-224c-4fe3-9daf-6de0a5fc0727 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.415334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23df61-09c0-4989-a9bd-9606698e0acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.415925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23df61-09c0-4989-a9bd-9606698e0acb admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.416204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23df61-09c0-4989-a9bd-9606698e0acb admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.416727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23df61-09c0-4989-a9bd-9606698e0acb admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.417441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23df61-09c0-4989-a9bd-9606698e0acb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.419570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23df61-09c0-4989-a9bd-9606698e0acb admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.449130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 791/1580] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:33 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:33.449936 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f27b71-dbc2-4850-be51-2bf137c62ab2 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.510809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f27b71-dbc2-4850-be51-2bf137c62ab2 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.515555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284078c-4e5e-4b11-aa8a-6ab12f9f124f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.516161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284078c-4e5e-4b11-aa8a-6ab12f9f124f admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.516975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284078c-4e5e-4b11-aa8a-6ab12f9f124f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.517267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284078c-4e5e-4b11-aa8a-6ab12f9f124f admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.517893 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284078c-4e5e-4b11-aa8a-6ab12f9f124f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.519825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284078c-4e5e-4b11-aa8a-6ab12f9f124f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.538445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 790/1581] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:33 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:33.538445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23df61-09c0-4989-a9bd-9606698e0acb admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.546444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23df61-09c0-4989-a9bd-9606698e0acb admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.551829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de06109-0cca-4795-9a8e-3636f8e93437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.551829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de06109-0cca-4795-9a8e-3636f8e93437 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.551829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de06109-0cca-4795-9a8e-3636f8e93437 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.552168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de06109-0cca-4795-9a8e-3636f8e93437 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.552781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de06109-0cca-4795-9a8e-3636f8e93437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.554363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de06109-0cca-4795-9a8e-3636f8e93437 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.564629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 791/1582] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:33.565366 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de06109-0cca-4795-9a8e-3636f8e93437 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.574480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de06109-0cca-4795-9a8e-3636f8e93437 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.583784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fd709-74b5-43b4-bf3d-35a6e5ae54cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.583784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fd709-74b5-43b4-bf3d-35a6e5ae54cb admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.583784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fd709-74b5-43b4-bf3d-35a6e5ae54cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.583784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fd709-74b5-43b4-bf3d-35a6e5ae54cb admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.584213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fd709-74b5-43b4-bf3d-35a6e5ae54cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.584213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fd709-74b5-43b4-bf3d-35a6e5ae54cb admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.614455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 792/1583] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:33 2022] POST /identity/v3/projects => generated 367 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:33.618215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284078c-4e5e-4b11-aa8a-6ab12f9f124f admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.661036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 792/1584] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:33 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:33.662106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fd709-74b5-43b4-bf3d-35a6e5ae54cb admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.676120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284078c-4e5e-4b11-aa8a-6ab12f9f124f admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.706711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:33.724268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fd709-74b5-43b4-bf3d-35a6e5ae54cb admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.729398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48297e-daa6-4430-afb4-f1028659509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.730023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48297e-daa6-4430-afb4-f1028659509d admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.730302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48297e-daa6-4430-afb4-f1028659509d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.730572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48297e-daa6-4430-afb4-f1028659509d admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.731208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48297e-daa6-4430-afb4-f1028659509d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.732780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48297e-daa6-4430-afb4-f1028659509d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.733733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c202d2-8231-49e0-bc31-662dd42af587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.734486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c202d2-8231-49e0-bc31-662dd42af587 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.738646 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c202d2-8231-49e0-bc31-662dd42af587 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.738646 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c202d2-8231-49e0-bc31-662dd42af587 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.738646 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c202d2-8231-49e0-bc31-662dd42af587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.738646 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c202d2-8231-49e0-bc31-662dd42af587 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.742948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 793/1585] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:33.743784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48297e-daa6-4430-afb4-f1028659509d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.751051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 793/1586] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:33.751950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c202d2-8231-49e0-bc31-662dd42af587 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.753303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48297e-daa6-4430-afb4-f1028659509d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.758024 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef053c-72b0-4912-80a2-eedf2a9e1c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.758607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef053c-72b0-4912-80a2-eedf2a9e1c2b admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.758883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef053c-72b0-4912-80a2-eedf2a9e1c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.759148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cef053c-72b0-4912-80a2-eedf2a9e1c2b admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.759753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef053c-72b0-4912-80a2-eedf2a9e1c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.761971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cef053c-72b0-4912-80a2-eedf2a9e1c2b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.762101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c202d2-8231-49e0-bc31-662dd42af587 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.767184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d819bc4-0734-4b72-8a52-125b584d3e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.767773 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d819bc4-0734-4b72-8a52-125b584d3e67 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.768049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d819bc4-0734-4b72-8a52-125b584d3e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.768317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d819bc4-0734-4b72-8a52-125b584d3e67 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.768928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d819bc4-0734-4b72-8a52-125b584d3e67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.770857 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d819bc4-0734-4b72-8a52-125b584d3e67 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.853733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 794/1587] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:33 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:33.855094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef053c-72b0-4912-80a2-eedf2a9e1c2b admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.864618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cef053c-72b0-4912-80a2-eedf2a9e1c2b admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.867237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 794/1588] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:33 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:33.867237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d819bc4-0734-4b72-8a52-125b584d3e67 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.869650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44b68c-6b13-4c7d-939b-a231ee7531cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.870269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44b68c-6b13-4c7d-939b-a231ee7531cb admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.870547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44b68c-6b13-4c7d-939b-a231ee7531cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.870821 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44b68c-6b13-4c7d-939b-a231ee7531cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.871447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44b68c-6b13-4c7d-939b-a231ee7531cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.873017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44b68c-6b13-4c7d-939b-a231ee7531cb admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.876432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d819bc4-0734-4b72-8a52-125b584d3e67 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.880934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf133f-0c29-4794-8bd4-0a83b7de0ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.881554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf133f-0c29-4794-8bd4-0a83b7de0ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.881826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf133f-0c29-4794-8bd4-0a83b7de0ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.882128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf133f-0c29-4794-8bd4-0a83b7de0ebe admin admin] PATH_INFO: `/v3/projects/fdeb9e648a4447d1b9ce7e035ca3be2e/users/84c46c4817184c9d8554c956bb7e48dc/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.884160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 795/1589] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:33.884919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44b68c-6b13-4c7d-939b-a231ee7531cb admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:33.895624 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44b68c-6b13-4c7d-939b-a231ee7531cb admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:33.949874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf133f-0c29-4794-8bd4-0a83b7de0ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdeb9e648a4447d1b9ce7e035ca3be2e, user_id=84c46c4817184c9d8554c956bb7e48dc, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.950043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394e0c1-8c9c-4093-92e8-1964076e1729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:33.950173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf133f-0c29-4794-8bd4-0a83b7de0ebe admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:33.950279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394e0c1-8c9c-4093-92e8-1964076e1729 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:33.950279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394e0c1-8c9c-4093-92e8-1964076e1729 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:33.950279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394e0c1-8c9c-4093-92e8-1964076e1729 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:33.950279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394e0c1-8c9c-4093-92e8-1964076e1729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:33.950279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394e0c1-8c9c-4093-92e8-1964076e1729 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.031343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 795/1590] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:33 2022] PUT /identity/v3/projects/fdeb9e648a4447d1b9ce7e035ca3be2e/users/84c46c4817184c9d8554c956bb7e48dc/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:34.032176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf133f-0c29-4794-8bd4-0a83b7de0ebe admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.042593 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 796/1591] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:33 2022] POST /identity/v3/users => generated 284 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:34.043594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394e0c1-8c9c-4093-92e8-1964076e1729 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.053339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394e0c1-8c9c-4093-92e8-1964076e1729 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.095407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf133f-0c29-4794-8bd4-0a83b7de0ebe admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.100321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c2f2c-d2e3-4df9-8c94-9106ade71093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.100914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c2f2c-d2e3-4df9-8c94-9106ade71093 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.101192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c2f2c-d2e3-4df9-8c94-9106ade71093 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.101463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c2f2c-d2e3-4df9-8c94-9106ade71093 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.102110 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c2f2c-d2e3-4df9-8c94-9106ade71093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.104097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8c2f2c-d2e3-4df9-8c94-9106ade71093 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.105716 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:34.112221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ce451-bf3b-4b7f-a597-cd66f3b69548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.112816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ce451-bf3b-4b7f-a597-cd66f3b69548 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.115866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ce451-bf3b-4b7f-a597-cd66f3b69548 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.115866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ce451-bf3b-4b7f-a597-cd66f3b69548 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.115866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ce451-bf3b-4b7f-a597-cd66f3b69548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.115866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ce451-bf3b-4b7f-a597-cd66f3b69548 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.125290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 797/1592] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:34.126588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ce451-bf3b-4b7f-a597-cd66f3b69548 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.136209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ce451-bf3b-4b7f-a597-cd66f3b69548 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.140738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e455c-f39b-4a8b-a595-9b083e0fcc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.141561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e455c-f39b-4a8b-a595-9b083e0fcc89 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.142364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e455c-f39b-4a8b-a595-9b083e0fcc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.142750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e455c-f39b-4a8b-a595-9b083e0fcc89 admin admin] PATH_INFO: `/v3/projects/c5aead202fae47d791fe9a58b8c8faf1/users/f4ee83b28c5f417792da4ab321a174d0/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.158239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e455c-f39b-4a8b-a595-9b083e0fcc89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5aead202fae47d791fe9a58b8c8faf1, user_id=f4ee83b28c5f417792da4ab321a174d0, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.160921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e455c-f39b-4a8b-a595-9b083e0fcc89 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.185047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 796/1593] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:34 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:34.189673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c2f2c-d2e3-4df9-8c94-9106ade71093 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.198039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c2f2c-d2e3-4df9-8c94-9106ade71093 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.202646 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a25c51-d76f-4edc-8b58-b10c618e86a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.203213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a25c51-d76f-4edc-8b58-b10c618e86a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.203488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a25c51-d76f-4edc-8b58-b10c618e86a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.203758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a25c51-d76f-4edc-8b58-b10c618e86a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.205216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a25c51-d76f-4edc-8b58-b10c618e86a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.206809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a25c51-d76f-4edc-8b58-b10c618e86a3 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.216577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 797/1594] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:34.217361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a25c51-d76f-4edc-8b58-b10c618e86a3 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.226651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a25c51-d76f-4edc-8b58-b10c618e86a3 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.230337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 798/1595] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:34 2022] PUT /identity/v3/projects/c5aead202fae47d791fe9a58b8c8faf1/users/f4ee83b28c5f417792da4ab321a174d0/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:34.237266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e455c-f39b-4a8b-a595-9b083e0fcc89 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.284495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f0c6c-e513-4d96-ba5f-ae9a70222567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.285196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f0c6c-e513-4d96-ba5f-ae9a70222567 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.285552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f0c6c-e513-4d96-ba5f-ae9a70222567 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.286038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f0c6c-e513-4d96-ba5f-ae9a70222567 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.286551 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f0c6c-e513-4d96-ba5f-ae9a70222567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.288128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f0c6c-e513-4d96-ba5f-ae9a70222567 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.296211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e455c-f39b-4a8b-a595-9b083e0fcc89 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.298880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 798/1596] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:34.300745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa714c-c592-4cfd-92f4-6039784339e7 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.301032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa714c-c592-4cfd-92f4-6039784339e7 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.301301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa714c-c592-4cfd-92f4-6039784339e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.304331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae002608-e0c2-400e-90cb-6c11b5e9f166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.305008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae002608-e0c2-400e-90cb-6c11b5e9f166 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.305356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae002608-e0c2-400e-90cb-6c11b5e9f166 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.305696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae002608-e0c2-400e-90cb-6c11b5e9f166 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.306434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae002608-e0c2-400e-90cb-6c11b5e9f166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.308105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae002608-e0c2-400e-90cb-6c11b5e9f166 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.309842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:34.319478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 799/1597] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:34.323366 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae002608-e0c2-400e-90cb-6c11b5e9f166 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.323477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-dffa714c-c592-4cfd-92f4-6039784339e7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:34.323943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-dffa714c-c592-4cfd-92f4-6039784339e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:34.330752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae002608-e0c2-400e-90cb-6c11b5e9f166 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.332261 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 799/1598] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:34.333212 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa714c-c592-4cfd-92f4-6039784339e7 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.336229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d94419-2c83-46f3-85ba-c3c89523ccf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.336828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94419-2c83-46f3-85ba-c3c89523ccf7 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.337137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94419-2c83-46f3-85ba-c3c89523ccf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.337408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94419-2c83-46f3-85ba-c3c89523ccf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.338077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d94419-2c83-46f3-85ba-c3c89523ccf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.339676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d94419-2c83-46f3-85ba-c3c89523ccf7 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.343067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa714c-c592-4cfd-92f4-6039784339e7 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.348962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb75f44-25f3-4912-9f69-30476850181b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.349620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb75f44-25f3-4912-9f69-30476850181b admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.349899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb75f44-25f3-4912-9f69-30476850181b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.350200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb75f44-25f3-4912-9f69-30476850181b admin admin] PATH_INFO: `/v3/projects/4388bc156f2b4c75a026a8d28a6127e7/users/6e8523a2227643cdb6071cca0bddcdc6/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.353015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 800/1599] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:34.353015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d94419-2c83-46f3-85ba-c3c89523ccf7 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.365816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d94419-2c83-46f3-85ba-c3c89523ccf7 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.368447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb75f44-25f3-4912-9f69-30476850181b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4388bc156f2b4c75a026a8d28a6127e7, user_id=6e8523a2227643cdb6071cca0bddcdc6, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.370595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc018c2a-4dfe-4402-bcc8-906bc9a462ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.371113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb75f44-25f3-4912-9f69-30476850181b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.371254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc018c2a-4dfe-4402-bcc8-906bc9a462ae admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.371490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc018c2a-4dfe-4402-bcc8-906bc9a462ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.371760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc018c2a-4dfe-4402-bcc8-906bc9a462ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.372392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc018c2a-4dfe-4402-bcc8-906bc9a462ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.373977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc018c2a-4dfe-4402-bcc8-906bc9a462ae admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.385975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 801/1600] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:34.386945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc018c2a-4dfe-4402-bcc8-906bc9a462ae admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.396413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc018c2a-4dfe-4402-bcc8-906bc9a462ae admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.401110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddf2ee-44ee-48a1-b96a-39973b06dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.401709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddf2ee-44ee-48a1-b96a-39973b06dd23 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.402013 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddf2ee-44ee-48a1-b96a-39973b06dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.402282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eddf2ee-44ee-48a1-b96a-39973b06dd23 admin admin] PATH_INFO: `/v3/projects/2f7ad8d1fa56413a894e461e35422074/users/1bdd7ff53de24613a215a51b6cf253ca/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.416253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddf2ee-44ee-48a1-b96a-39973b06dd23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7ad8d1fa56413a894e461e35422074, user_id=1bdd7ff53de24613a215a51b6cf253ca, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.419211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eddf2ee-44ee-48a1-b96a-39973b06dd23 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.436325 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 800/1601] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:34 2022] PUT /identity/v3/projects/4388bc156f2b4c75a026a8d28a6127e7/users/6e8523a2227643cdb6071cca0bddcdc6/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:34.437561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c824ea8-3e87-45f4-b086-cf349f193c87 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.437989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c824ea8-3e87-45f4-b086-cf349f193c87 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.438267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c824ea8-3e87-45f4-b086-cf349f193c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.458221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-3c824ea8-3e87-45f4-b086-cf349f193c87 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:34.459564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3c824ea8-3e87-45f4-b086-cf349f193c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:34.517715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 801/1602] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:34.518597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c824ea8-3e87-45f4-b086-cf349f193c87 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.531429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c824ea8-3e87-45f4-b086-cf349f193c87 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.539012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 802/1603] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:34 2022] PUT /identity/v3/projects/2f7ad8d1fa56413a894e461e35422074/users/1bdd7ff53de24613a215a51b6cf253ca/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:34.539961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddf2ee-44ee-48a1-b96a-39973b06dd23 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.595683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee304bf5-ac9e-46c2-83d0-a7756b3e9948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.596397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee304bf5-ac9e-46c2-83d0-a7756b3e9948 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.596677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee304bf5-ac9e-46c2-83d0-a7756b3e9948 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.596974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee304bf5-ac9e-46c2-83d0-a7756b3e9948 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.597810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee304bf5-ac9e-46c2-83d0-a7756b3e9948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.599444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee304bf5-ac9e-46c2-83d0-a7756b3e9948 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.609359 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 802/1604] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:34.610846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee304bf5-ac9e-46c2-83d0-a7756b3e9948 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.611919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eddf2ee-44ee-48a1-b96a-39973b06dd23 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.616896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54400ddb-bde0-4784-93e8-678790dbbded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.617518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54400ddb-bde0-4784-93e8-678790dbbded admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.617819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54400ddb-bde0-4784-93e8-678790dbbded admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.618180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54400ddb-bde0-4784-93e8-678790dbbded admin admin] PATH_INFO: `/v3/projects/01ca6df74bca45fc9edde426ba6ee069/users/be94daa21a15438f8ff005e44b133363/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.633806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54400ddb-bde0-4784-93e8-678790dbbded admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ca6df74bca45fc9edde426ba6ee069, user_id=be94daa21a15438f8ff005e44b133363, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.636465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54400ddb-bde0-4784-93e8-678790dbbded admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.664408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee304bf5-ac9e-46c2-83d0-a7756b3e9948 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.668511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e573394-8567-433d-8133-c2c427480547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.669337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e573394-8567-433d-8133-c2c427480547 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.669614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e573394-8567-433d-8133-c2c427480547 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.669886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e573394-8567-433d-8133-c2c427480547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.679514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e573394-8567-433d-8133-c2c427480547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.680916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e573394-8567-433d-8133-c2c427480547 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.690376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 803/1605] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:34.691133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e573394-8567-433d-8133-c2c427480547 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.700011 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e573394-8567-433d-8133-c2c427480547 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.705005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e93097-1f83-4715-9473-97b40bddeca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.707657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e93097-1f83-4715-9473-97b40bddeca0 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.707657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e93097-1f83-4715-9473-97b40bddeca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.707657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e93097-1f83-4715-9473-97b40bddeca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.707657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e93097-1f83-4715-9473-97b40bddeca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.708345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e93097-1f83-4715-9473-97b40bddeca0 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.713790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:34.718713 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 804/1606] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:34.720163 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e93097-1f83-4715-9473-97b40bddeca0 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.729392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e93097-1f83-4715-9473-97b40bddeca0 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.733909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b929db8-d3da-43ea-905d-a9675e1f582d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.734490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b929db8-d3da-43ea-905d-a9675e1f582d admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.734878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b929db8-d3da-43ea-905d-a9675e1f582d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.735173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b929db8-d3da-43ea-905d-a9675e1f582d admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.736319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b929db8-d3da-43ea-905d-a9675e1f582d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.737924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b929db8-d3da-43ea-905d-a9675e1f582d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.742097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:34.746019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 803/1607] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:34 2022] PUT /identity/v3/projects/01ca6df74bca45fc9edde426ba6ee069/users/be94daa21a15438f8ff005e44b133363/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:34.747239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee4fc1-e940-4a50-a9d5-5cc933d26477 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.747756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee4fc1-e940-4a50-a9d5-5cc933d26477 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.748018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee4fc1-e940-4a50-a9d5-5cc933d26477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.752098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 805/1608] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:34.752098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b929db8-d3da-43ea-905d-a9675e1f582d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.759181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b929db8-d3da-43ea-905d-a9675e1f582d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.776439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-65ee4fc1-e940-4a50-a9d5-5cc933d26477 None None] MFA Rules not processed for user `6e8523a2227643cdb6071cca0bddcdc6`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:34.777636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-65ee4fc1-e940-4a50-a9d5-5cc933d26477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:34.808440 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b138945-e95e-4c4e-bb73-cee6a710500b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.809031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b138945-e95e-4c4e-bb73-cee6a710500b service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.809300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b138945-e95e-4c4e-bb73-cee6a710500b service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.809565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b138945-e95e-4c4e-bb73-cee6a710500b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.842802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 804/1609] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:34.843845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee4fc1-e940-4a50-a9d5-5cc933d26477 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.894799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b138945-e95e-4c4e-bb73-cee6a710500b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.896424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b138945-e95e-4c4e-bb73-cee6a710500b service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.907889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 806/1610] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:34.908650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b138945-e95e-4c4e-bb73-cee6a710500b service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.918344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b138945-e95e-4c4e-bb73-cee6a710500b service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.924229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37e944-42b9-4802-8e00-18b0d115c896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.924855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37e944-42b9-4802-8e00-18b0d115c896 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.925170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37e944-42b9-4802-8e00-18b0d115c896 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.925446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37e944-42b9-4802-8e00-18b0d115c896 admin admin] PATH_INFO: `/v3/projects/06c88ae5d5914faba8ff52380a8fcafa/users/6b7457ab864e4df6b10a195f8500c14c/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.926300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee4fc1-e940-4a50-a9d5-5cc933d26477 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.931856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e352f-613a-400c-b639-0853db15593d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.932462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e352f-613a-400c-b639-0853db15593d admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.932736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e352f-613a-400c-b639-0853db15593d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.933000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e352f-613a-400c-b639-0853db15593d admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.933631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e352f-613a-400c-b639-0853db15593d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.935271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e352f-613a-400c-b639-0853db15593d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.945079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37e944-42b9-4802-8e00-18b0d115c896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c88ae5d5914faba8ff52380a8fcafa, user_id=6b7457ab864e4df6b10a195f8500c14c, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.947866 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37e944-42b9-4802-8e00-18b0d115c896 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:34.948701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 805/1611] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:34.949510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e352f-613a-400c-b639-0853db15593d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:34.959301 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e352f-613a-400c-b639-0853db15593d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:34.963829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163fb3c-d369-42b6-9d67-0da36f8960a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:34.964451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163fb3c-d369-42b6-9d67-0da36f8960a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:34.964722 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163fb3c-d369-42b6-9d67-0da36f8960a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:34.964986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163fb3c-d369-42b6-9d67-0da36f8960a4 admin admin] PATH_INFO: `/v3/projects/8c56db670e834c97b57386cd583ca59f/users/0d46ec7a77eb419e9686cc88245ec288/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:34.979670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163fb3c-d369-42b6-9d67-0da36f8960a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c56db670e834c97b57386cd583ca59f, user_id=0d46ec7a77eb419e9686cc88245ec288, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:34.982293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163fb3c-d369-42b6-9d67-0da36f8960a4 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:35.042181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 807/1612] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:34 2022] PUT /identity/v3/projects/06c88ae5d5914faba8ff52380a8fcafa/users/6b7457ab864e4df6b10a195f8500c14c/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:35.042181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc8cbb-a44d-46dd-8bd2-fdef4f048d87 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.042181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc8cbb-a44d-46dd-8bd2-fdef4f048d87 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.042181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc8cbb-a44d-46dd-8bd2-fdef4f048d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.059709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-85bc8cbb-a44d-46dd-8bd2-fdef4f048d87 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:35.061088 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-85bc8cbb-a44d-46dd-8bd2-fdef4f048d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:35.066347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 806/1613] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:34 2022] PUT /identity/v3/projects/8c56db670e834c97b57386cd583ca59f/users/0d46ec7a77eb419e9686cc88245ec288/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:35.069038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163fb3c-d369-42b6-9d67-0da36f8960a4 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:35.131241 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 808/1614] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:35.132707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e3017-0de9-421a-88ca-df8c1ad6e799 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.132989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e3017-0de9-421a-88ca-df8c1ad6e799 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.133258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e3017-0de9-421a-88ca-df8c1ad6e799 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.141202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163fb3c-d369-42b6-9d67-0da36f8960a4 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:35.142636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 809/1615] 10.222.0.79 () {54 vars in 962 bytes} [Tue Aug 9 16:51:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:51:35.143772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e3017-0de9-421a-88ca-df8c1ad6e799 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:35.147538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9d595-cac3-47a5-9cc0-ed0f1882ffed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:35.148249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9d595-cac3-47a5-9cc0-ed0f1882ffed admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.148623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9d595-cac3-47a5-9cc0-ed0f1882ffed admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.148927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9d595-cac3-47a5-9cc0-ed0f1882ffed admin admin] PATH_INFO: `/v3/projects/b44445ab9759401083af9109bfb04fed/users/ccad0b7ad5274d02b43f9c2fb857452a/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.158363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e3017-0de9-421a-88ca-df8c1ad6e799 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:35.163619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84c12a-ec60-46c7-bcb4-19980c788d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:35.164269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84c12a-ec60-46c7-bcb4-19980c788d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.164620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84c12a-ec60-46c7-bcb4-19980c788d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.165127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84c12a-ec60-46c7-bcb4-19980c788d88 admin admin] PATH_INFO: `/v3/projects/2f7ad8d1fa56413a894e461e35422074/users/1bdd7ff53de24613a215a51b6cf253ca/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.166188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9d595-cac3-47a5-9cc0-ed0f1882ffed admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44445ab9759401083af9109bfb04fed, user_id=ccad0b7ad5274d02b43f9c2fb857452a, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:35.167402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84c12a-ec60-46c7-bcb4-19980c788d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7ad8d1fa56413a894e461e35422074, user_id=1bdd7ff53de24613a215a51b6cf253ca, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:35.169403 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea9d595-cac3-47a5-9cc0-ed0f1882ffed admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:35.170225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e84c12a-ec60-46c7-bcb4-19980c788d88 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:35.182157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 810/1616] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:35 2022] PUT /identity/v3/projects/2f7ad8d1fa56413a894e461e35422074/users/1bdd7ff53de24613a215a51b6cf253ca/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:35.183490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636afe9-acc7-42e0-86bf-f5d42cd9a7cb None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.183836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636afe9-acc7-42e0-86bf-f5d42cd9a7cb None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.184170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636afe9-acc7-42e0-86bf-f5d42cd9a7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.197330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:35.211230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c636afe9-acc7-42e0-86bf-f5d42cd9a7cb None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:35.212473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c636afe9-acc7-42e0-86bf-f5d42cd9a7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:35.244001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 807/1617] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:35 2022] PUT /identity/v3/projects/b44445ab9759401083af9109bfb04fed/users/ccad0b7ad5274d02b43f9c2fb857452a/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:35.245890 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c90874-00a0-4307-b4a6-bbf851c480f3 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.246268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c90874-00a0-4307-b4a6-bbf851c480f3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.246603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c90874-00a0-4307-b4a6-bbf851c480f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.278174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-71c90874-00a0-4307-b4a6-bbf851c480f3 None None] MFA Rules not processed for user `0d46ec7a77eb419e9686cc88245ec288`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:35.279431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-71c90874-00a0-4307-b4a6-bbf851c480f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:35.281627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 811/1618] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:35.282544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636afe9-acc7-42e0-86bf-f5d42cd9a7cb None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:35.312380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:35.337120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636afe9-acc7-42e0-86bf-f5d42cd9a7cb None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:35.351990 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 808/1619] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:35.353483 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0a6e1-b6e6-4619-8106-a55d5497b0b9 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.354175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0a6e1-b6e6-4619-8106-a55d5497b0b9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.354600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0a6e1-b6e6-4619-8106-a55d5497b0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.399438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316019e-4b51-4f39-93c5-e0f71815a532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:35.400271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316019e-4b51-4f39-93c5-e0f71815a532 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.400701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316019e-4b51-4f39-93c5-e0f71815a532 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.400876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b4b0a6e1-b6e6-4619-8106-a55d5497b0b9 None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:35.401102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316019e-4b51-4f39-93c5-e0f71815a532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.402387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b4b0a6e1-b6e6-4619-8106-a55d5497b0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:35.481240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 809/1620] 10.222.0.79 () {58 vars in 1059 bytes} [Tue Aug 9 16:51:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:35.482896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474a7b3-45f4-4979-9d62-b59152c1e5ac None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.483232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474a7b3-45f4-4979-9d62-b59152c1e5ac None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.488380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474a7b3-45f4-4979-9d62-b59152c1e5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.490988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316019e-4b51-4f39-93c5-e0f71815a532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:35.492679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5316019e-4b51-4f39-93c5-e0f71815a532 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:35.509258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 812/1621] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:35.510715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf40d42-4e08-408e-9138-f40dae5dacd8 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.511068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf40d42-4e08-408e-9138-f40dae5dacd8 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.511597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf40d42-4e08-408e-9138-f40dae5dacd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.540147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-7474a7b3-45f4-4979-9d62-b59152c1e5ac None None] MFA Rules not processed for user `1bdd7ff53de24613a215a51b6cf253ca`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:35.541720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-7474a7b3-45f4-4979-9d62-b59152c1e5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:35.556676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-bbf40d42-4e08-408e-9138-f40dae5dacd8 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:35.558080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bbf40d42-4e08-408e-9138-f40dae5dacd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:35.567798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 813/1622] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:35.568566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf40d42-4e08-408e-9138-f40dae5dacd8 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:35.586966 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf40d42-4e08-408e-9138-f40dae5dacd8 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:35.591356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8cf13-0dcc-4e0d-9d9b-53ef3e3451c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:35.591974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8cf13-0dcc-4e0d-9d9b-53ef3e3451c8 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.592252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8cf13-0dcc-4e0d-9d9b-53ef3e3451c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.592522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8cf13-0dcc-4e0d-9d9b-53ef3e3451c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.619919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8cf13-0dcc-4e0d-9d9b-53ef3e3451c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:35.622493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8cf13-0dcc-4e0d-9d9b-53ef3e3451c8 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:35.639405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 810/1623] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:35.640727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474a7b3-45f4-4979-9d62-b59152c1e5ac None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:35.643389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 814/1624] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:35.644910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8cf13-0dcc-4e0d-9d9b-53ef3e3451c8 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:35.663335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8cf13-0dcc-4e0d-9d9b-53ef3e3451c8 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:35.667162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a9df0-85f2-4d16-bf6c-678cef33c16f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:35.667777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a9df0-85f2-4d16-bf6c-678cef33c16f service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.668188 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a9df0-85f2-4d16-bf6c-678cef33c16f service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.668365 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a9df0-85f2-4d16-bf6c-678cef33c16f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.708434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474a7b3-45f4-4979-9d62-b59152c1e5ac None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:35.714093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e26e37-d2cf-4341-982a-ead55e9ed508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:35.718226 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e26e37-d2cf-4341-982a-ead55e9ed508 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.718647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e26e37-d2cf-4341-982a-ead55e9ed508 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.719072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e26e37-d2cf-4341-982a-ead55e9ed508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.730586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e26e37-d2cf-4341-982a-ead55e9ed508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:35.732185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e26e37-d2cf-4341-982a-ead55e9ed508 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:35.743947 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a9df0-85f2-4d16-bf6c-678cef33c16f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:35.746074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a9df0-85f2-4d16-bf6c-678cef33c16f service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:35.748940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 811/1625] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:35.749911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e26e37-d2cf-4341-982a-ead55e9ed508 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:35.756938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 815/1626] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:35.764016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e26e37-d2cf-4341-982a-ead55e9ed508 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:35.771258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a9df0-85f2-4d16-bf6c-678cef33c16f service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:35.774868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec3fbf-d028-4bf6-8c69-ac9c75b36881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:35.776790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3fbf-d028-4bf6-8c69-ac9c75b36881 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.777099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3fbf-d028-4bf6-8c69-ac9c75b36881 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.777380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec3fbf-d028-4bf6-8c69-ac9c75b36881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.789661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a9df0-85f2-4d16-bf6c-678cef33c16f service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:35.797635 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec3fbf-d028-4bf6-8c69-ac9c75b36881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:35.800949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec3fbf-d028-4bf6-8c69-ac9c75b36881 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:35.802537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e3d1aa-c2d0-4ffe-8d00-5ea481aa6a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:35.803254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3d1aa-c2d0-4ffe-8d00-5ea481aa6a4d service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:35.803633 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3d1aa-c2d0-4ffe-8d00-5ea481aa6a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:35.804108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e3d1aa-c2d0-4ffe-8d00-5ea481aa6a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:35.815453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3d1aa-c2d0-4ffe-8d00-5ea481aa6a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:35.818788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e3d1aa-c2d0-4ffe-8d00-5ea481aa6a4d service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:35.823801 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 812/1627] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:35.835224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 816/1628] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:35.900459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:36.019360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:36.211616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec3fbf-d028-4bf6-8c69-ac9c75b36881 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:36.224514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec3fbf-d028-4bf6-8c69-ac9c75b36881 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:36.229547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482107f-dda1-4901-be29-97550169c464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:36.231848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482107f-dda1-4901-be29-97550169c464 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:36.231848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482107f-dda1-4901-be29-97550169c464 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:36.231848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482107f-dda1-4901-be29-97550169c464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:36.313990 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482107f-dda1-4901-be29-97550169c464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:36.316015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482107f-dda1-4901-be29-97550169c464 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:36.364025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 813/1629] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:36.397340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e05ed-320c-421d-b6b5-dcbcda837a87 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:36.397626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e05ed-320c-421d-b6b5-dcbcda837a87 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:36.399559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e05ed-320c-421d-b6b5-dcbcda837a87 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:36.399559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 817/1630] 10.222.0.79 () {54 vars in 962 bytes} [Tue Aug 9 16:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:51:36.406928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab3b5f-3832-4c0a-847a-ef4593d0f2d0 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:36.407464 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab3b5f-3832-4c0a-847a-ef4593d0f2d0 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:36.407891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab3b5f-3832-4c0a-847a-ef4593d0f2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:36.443774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f2ab3b5f-3832-4c0a-847a-ef4593d0f2d0 None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:36.446950 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f2ab3b5f-3832-4c0a-847a-ef4593d0f2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:36.464181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 814/1631] 10.222.0.79 () {58 vars in 1059 bytes} [Tue Aug 9 16:51:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:36.476572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754633c9-9aad-4daf-a4f0-f3b76bb18a2e None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:36.476864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754633c9-9aad-4daf-a4f0-f3b76bb18a2e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:36.477140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754633c9-9aad-4daf-a4f0-f3b76bb18a2e None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:36.478806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 818/1632] 10.222.0.79 () {54 vars in 912 bytes} [Tue Aug 9 16:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:51:36.485528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5133b6c-7d9f-4f2b-87a8-31e7cfe902ee None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:36.486200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5133b6c-7d9f-4f2b-87a8-31e7cfe902ee None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:36.486583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5133b6c-7d9f-4f2b-87a8-31e7cfe902ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:36.513266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f5133b6c-7d9f-4f2b-87a8-31e7cfe902ee None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:36.515081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f5133b6c-7d9f-4f2b-87a8-31e7cfe902ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:36.534995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 815/1633] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 16:51:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:51:36.851769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754633c9-9aad-4daf-a4f0-f3b76bb18a2e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:36.864938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754633c9-9aad-4daf-a4f0-f3b76bb18a2e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:36.869138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b6762-4c01-41fc-84e1-d4cc46a2a744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:36.869824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b6762-4c01-41fc-84e1-d4cc46a2a744 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:36.870190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b6762-4c01-41fc-84e1-d4cc46a2a744 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:36.870797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b6762-4c01-41fc-84e1-d4cc46a2a744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:36.903361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b6762-4c01-41fc-84e1-d4cc46a2a744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:36.906724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b6762-4c01-41fc-84e1-d4cc46a2a744 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:36.932981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 819/1634] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:37.099239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac41862-a270-4ce5-9630-7686df22661a None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:37.104089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac41862-a270-4ce5-9630-7686df22661a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:37.104089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac41862-a270-4ce5-9630-7686df22661a None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:37.104089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 816/1635] 10.222.0.79 () {54 vars in 912 bytes} [Tue Aug 9 16:51:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:51:37.109083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3403df8-6007-4b05-81a8-c23616605b1f None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:37.109561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3403df8-6007-4b05-81a8-c23616605b1f None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:37.109907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3403df8-6007-4b05-81a8-c23616605b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:37.146231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-f3403df8-6007-4b05-81a8-c23616605b1f None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:37.147543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-f3403df8-6007-4b05-81a8-c23616605b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:37.170181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 820/1636] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 16:51:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:37.187382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac41862-a270-4ce5-9630-7686df22661a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:37.208925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac41862-a270-4ce5-9630-7686df22661a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:37.219901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd320ca3-48db-4742-96af-feb69c16a22e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:37.220647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd320ca3-48db-4742-96af-feb69c16a22e service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:37.220931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd320ca3-48db-4742-96af-feb69c16a22e service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:37.221202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd320ca3-48db-4742-96af-feb69c16a22e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:37.238246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd320ca3-48db-4742-96af-feb69c16a22e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:37.240031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd320ca3-48db-4742-96af-feb69c16a22e service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:37.258183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 817/1637] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:37.417411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3403df8-6007-4b05-81a8-c23616605b1f None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:37.444399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3403df8-6007-4b05-81a8-c23616605b1f None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:37.450250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:37.451066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:37.451949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:37.452418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] PATH_INFO: `/v3/users/6e8523a2227643cdb6071cca0bddcdc6` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:37.455599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8523a2227643cdb6071cca0bddcdc6)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:37.458607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:37.707911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8523a2227643cdb6071cca0bddcdc6'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:51:37.807592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8523a2227643cdb6071cca0bddcdc6'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:51:37.819109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8523a2227643cdb6071cca0bddcdc6'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:51:37.845587 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8523a2227643cdb6071cca0bddcdc6'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:51:37.846465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8523a2227643cdb6071cca0bddcdc6'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:51:37.853179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 821/1638] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:51:37 2022] DELETE /identity/v3/users/6e8523a2227643cdb6071cca0bddcdc6 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:37.857624 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb75ede-403a-445c-8d94-4e86419261ff None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:37.862911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb75ede-403a-445c-8d94-4e86419261ff None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:37.862911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb75ede-403a-445c-8d94-4e86419261ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:37.890656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ebb75ede-403a-445c-8d94-4e86419261ff None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:37.892264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ebb75ede-403a-445c-8d94-4e86419261ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:37.945435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:37.998541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 818/1639] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:38.012514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:38.013734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:38.053771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:38.338768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bffa95-4589-4054-9a9e-6655e5ce9a00 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:38.338768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097baba-6ad8-4002-b080-55876b0a6e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:38.338768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097baba-6ad8-4002-b080-55876b0a6e23 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:38.338768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097baba-6ad8-4002-b080-55876b0a6e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:38.340232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097baba-6ad8-4002-b080-55876b0a6e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:38.340232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097baba-6ad8-4002-b080-55876b0a6e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:38.340232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097baba-6ad8-4002-b080-55876b0a6e23 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:38.340232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 822/1640] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:40.035603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb75ede-403a-445c-8d94-4e86419261ff None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:40.037257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ebb75ede-403a-445c-8d94-4e86419261ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:40.061463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb75ede-403a-445c-8d94-4e86419261ff None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:40.079058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:40.079668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:40.079946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:40.080214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] PATH_INFO: `/v3/projects/4388bc156f2b4c75a026a8d28a6127e7` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:40.081501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4388bc156f2b4c75a026a8d28a6127e7)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:40.083435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:40.287455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4388bc156f2b4c75a026a8d28a6127e7', 'request_id': 'req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:51:40.355986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4388bc156f2b4c75a026a8d28a6127e7', 'request_id': 'req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:51:40.375494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4388bc156f2b4c75a026a8d28a6127e7', 'request_id': 'req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:51:40.404263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] The token cache is being invalidate because project 4388bc156f2b4c75a026a8d28a6127e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:51:40.404704 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:51:40.406770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7cc3a0d6-1596-4072-a89b-ceb3dd5681b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:51:40.415904 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 819/1641] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:51:40 2022] DELETE /identity/v3/projects/4388bc156f2b4c75a026a8d28a6127e7 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:40.418020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7cc88-8db1-4016-af7b-70c16d1e3a49 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:40.418493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7cc88-8db1-4016-af7b-70c16d1e3a49 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:40.418764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7cc88-8db1-4016-af7b-70c16d1e3a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:40.480479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-cfb7cc88-8db1-4016-af7b-70c16d1e3a49 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:40.487126 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cfb7cc88-8db1-4016-af7b-70c16d1e3a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:40.633224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 823/1642] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:40.643073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed144681-3211-4b36-87d0-643c12b093c5 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:40.643073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed144681-3211-4b36-87d0-643c12b093c5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:40.643073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed144681-3211-4b36-87d0-643c12b093c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:40.695732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ed144681-3211-4b36-87d0-643c12b093c5 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:40.697001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ed144681-3211-4b36-87d0-643c12b093c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:40.736316 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 820/1643] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:40.740729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7cc88-8db1-4016-af7b-70c16d1e3a49 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:40.754023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb7cc88-8db1-4016-af7b-70c16d1e3a49 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:40.761904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2c4d8-b3d2-4469-8b04-2389e48b5e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:40.764963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2c4d8-b3d2-4469-8b04-2389e48b5e21 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:40.764963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2c4d8-b3d2-4469-8b04-2389e48b5e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:40.764963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2c4d8-b3d2-4469-8b04-2389e48b5e21 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:40.765769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2c4d8-b3d2-4469-8b04-2389e48b5e21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:51:40.787387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2c4d8-b3d2-4469-8b04-2389e48b5e21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:40.790477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2c4d8-b3d2-4469-8b04-2389e48b5e21 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:40.815405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 824/1644] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:51:40.819916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed144681-3211-4b36-87d0-643c12b093c5 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:40.853266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed144681-3211-4b36-87d0-643c12b093c5 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:40.861387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187900c-6344-4760-ba94-ea14bfd7d19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:40.862369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187900c-6344-4760-ba94-ea14bfd7d19c admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:40.862940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187900c-6344-4760-ba94-ea14bfd7d19c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:40.863422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187900c-6344-4760-ba94-ea14bfd7d19c admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:40.864244 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187900c-6344-4760-ba94-ea14bfd7d19c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:40.867043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187900c-6344-4760-ba94-ea14bfd7d19c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:40.988419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 821/1645] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:40 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:40.993004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2c4d8-b3d2-4469-8b04-2389e48b5e21 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:41.020488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2c4d8-b3d2-4469-8b04-2389e48b5e21 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:41.173717 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d575a-46a3-4834-a55b-57f9d48fde58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:41.174723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d575a-46a3-4834-a55b-57f9d48fde58 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:41.175144 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d575a-46a3-4834-a55b-57f9d48fde58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:41.175801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d575a-46a3-4834-a55b-57f9d48fde58 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:41.176599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d575a-46a3-4834-a55b-57f9d48fde58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:41.180672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d575a-46a3-4834-a55b-57f9d48fde58 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:41.312359 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 825/1646] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:40 2022] POST /identity/v3/users => generated 287 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:41.317231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187900c-6344-4760-ba94-ea14bfd7d19c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:41.333856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187900c-6344-4760-ba94-ea14bfd7d19c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:41.341006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbfd98-bd6d-4796-8575-109029b23c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:41.342188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbfd98-bd6d-4796-8575-109029b23c9c admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:41.357332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbfd98-bd6d-4796-8575-109029b23c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:41.360023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbfd98-bd6d-4796-8575-109029b23c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:41.361489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbfd98-bd6d-4796-8575-109029b23c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:41.368293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbfd98-bd6d-4796-8575-109029b23c9c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:41.394393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 822/1647] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:41.399171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d575a-46a3-4834-a55b-57f9d48fde58 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:41.416723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d575a-46a3-4834-a55b-57f9d48fde58 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:41.425370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b996373-454c-4ba6-b78e-c8ecf28761bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:41.429068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b996373-454c-4ba6-b78e-c8ecf28761bd admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:41.429736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b996373-454c-4ba6-b78e-c8ecf28761bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:41.430438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b996373-454c-4ba6-b78e-c8ecf28761bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:41.431317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b996373-454c-4ba6-b78e-c8ecf28761bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:41.433540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b996373-454c-4ba6-b78e-c8ecf28761bd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:41.504845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbfd98-bd6d-4796-8575-109029b23c9c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:41.510117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 826/1648] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:41.540465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbfd98-bd6d-4796-8575-109029b23c9c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:41.553273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d1937-beb5-4d51-870d-81c959a1dd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:41.556241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d1937-beb5-4d51-870d-81c959a1dd81 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:41.556743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d1937-beb5-4d51-870d-81c959a1dd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:41.557233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d1937-beb5-4d51-870d-81c959a1dd81 admin admin] PATH_INFO: `/v3/projects/cff6c5637d4b49a6a40719df731e3193/users/e0d71ca66aa84022a1481e678b7a9484/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:41.637686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d1937-beb5-4d51-870d-81c959a1dd81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff6c5637d4b49a6a40719df731e3193, user_id=e0d71ca66aa84022a1481e678b7a9484, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:41.640849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d1937-beb5-4d51-870d-81c959a1dd81 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:41.758763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309ad2-66ce-47f2-a4ac-ae417d869071 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:41.759671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309ad2-66ce-47f2-a4ac-ae417d869071 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:41.760141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309ad2-66ce-47f2-a4ac-ae417d869071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:41.763818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 823/1649] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:41 2022] PUT /identity/v3/projects/cff6c5637d4b49a6a40719df731e3193/users/e0d71ca66aa84022a1481e678b7a9484/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:41.820050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-37309ad2-66ce-47f2-a4ac-ae417d869071 None None] MFA Rules not processed for user `e0d71ca66aa84022a1481e678b7a9484`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:41.825310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-37309ad2-66ce-47f2-a4ac-ae417d869071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:41.902639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:42.035017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d1937-beb5-4d51-870d-81c959a1dd81 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:42.039981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 827/1650] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:42.198462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d1937-beb5-4d51-870d-81c959a1dd81 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:42.204396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf55cde-6366-437b-9018-c486f3b6b1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:42.205449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf55cde-6366-437b-9018-c486f3b6b1c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:42.206552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf55cde-6366-437b-9018-c486f3b6b1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:42.206936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf55cde-6366-437b-9018-c486f3b6b1c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:42.207850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf55cde-6366-437b-9018-c486f3b6b1c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:42.211003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf55cde-6366-437b-9018-c486f3b6b1c5 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:42.217637 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:42.341017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 824/1651] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:51:42 2022] POST /identity/v3/projects => generated 373 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:42.346891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309ad2-66ce-47f2-a4ac-ae417d869071 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:42.363552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309ad2-66ce-47f2-a4ac-ae417d869071 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:42.575103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211f625-652e-45c0-9945-967747bbe75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:42.576353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211f625-652e-45c0-9945-967747bbe75d admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:42.576797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211f625-652e-45c0-9945-967747bbe75d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:42.577206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211f625-652e-45c0-9945-967747bbe75d admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:42.578012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211f625-652e-45c0-9945-967747bbe75d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:42.580502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211f625-652e-45c0-9945-967747bbe75d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:42.743900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf55cde-6366-437b-9018-c486f3b6b1c5 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:42.744318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 828/1652] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:51:42 2022] POST /identity/v3/users => generated 293 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:51:42.764454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf55cde-6366-437b-9018-c486f3b6b1c5 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:42.784745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce6605-ab38-45b0-9098-2d21fbbbc6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:42.785401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce6605-ab38-45b0-9098-2d21fbbbc6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:42.785675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce6605-ab38-45b0-9098-2d21fbbbc6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:42.785940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce6605-ab38-45b0-9098-2d21fbbbc6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:42.789566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce6605-ab38-45b0-9098-2d21fbbbc6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:42.789566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ce6605-ab38-45b0-9098-2d21fbbbc6a9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:42.821533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211f625-652e-45c0-9945-967747bbe75d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:42.831514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 825/1653] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:42.833727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211f625-652e-45c0-9945-967747bbe75d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:42.857527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93906610-f2e7-4f72-b249-8585c4ad69ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:42.858264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93906610-f2e7-4f72-b249-8585c4ad69ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:42.858543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93906610-f2e7-4f72-b249-8585c4ad69ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:42.858813 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93906610-f2e7-4f72-b249-8585c4ad69ec admin admin] PATH_INFO: `/v3/projects/118677e72243477e87b5049ba53cb683/users/9acb41eec4d546408dae9f360ba98942/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:42.910795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93906610-f2e7-4f72-b249-8585c4ad69ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=118677e72243477e87b5049ba53cb683, user_id=9acb41eec4d546408dae9f360ba98942, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:42.920408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93906610-f2e7-4f72-b249-8585c4ad69ec admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:43.076417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce6605-ab38-45b0-9098-2d21fbbbc6a9 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:43.083211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 829/1654] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:42 2022] PUT /identity/v3/projects/118677e72243477e87b5049ba53cb683/users/9acb41eec4d546408dae9f360ba98942/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:51:43.096373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ce6605-ab38-45b0-9098-2d21fbbbc6a9 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:43.247532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0f5fe-67da-4341-a739-198f7de1230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:43.248587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0f5fe-67da-4341-a739-198f7de1230a admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:43.252152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0f5fe-67da-4341-a739-198f7de1230a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:43.252746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0f5fe-67da-4341-a739-198f7de1230a admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:43.255019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0f5fe-67da-4341-a739-198f7de1230a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:43.258000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0f5fe-67da-4341-a739-198f7de1230a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:43.283981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 826/1655] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:51:43.289113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93906610-f2e7-4f72-b249-8585c4ad69ec admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:43.308122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93906610-f2e7-4f72-b249-8585c4ad69ec admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:43.318720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86ecb4-b412-447f-8b3c-e49994e54870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:43.320296 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86ecb4-b412-447f-8b3c-e49994e54870 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:43.322972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86ecb4-b412-447f-8b3c-e49994e54870 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:43.322972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86ecb4-b412-447f-8b3c-e49994e54870 admin admin] PATH_INFO: `/v3/projects/118677e72243477e87b5049ba53cb683/users/9acb41eec4d546408dae9f360ba98942/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:43.324074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86ecb4-b412-447f-8b3c-e49994e54870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=118677e72243477e87b5049ba53cb683, user_id=9acb41eec4d546408dae9f360ba98942, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:43.327116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86ecb4-b412-447f-8b3c-e49994e54870 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:43.361790 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 830/1656] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:51:43 2022] PUT /identity/v3/projects/118677e72243477e87b5049ba53cb683/users/9acb41eec4d546408dae9f360ba98942/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:51:43.374351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125eaaf-bc1e-415d-9ef2-22dfb11fae40 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:43.374921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125eaaf-bc1e-415d-9ef2-22dfb11fae40 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:43.374921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1125eaaf-bc1e-415d-9ef2-22dfb11fae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:43.414953 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-1125eaaf-bc1e-415d-9ef2-22dfb11fae40 None None] MFA Rules not processed for user `9acb41eec4d546408dae9f360ba98942`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:43.424797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-1125eaaf-bc1e-415d-9ef2-22dfb11fae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:43.545280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 827/1657] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:43.583249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86ecb4-b412-447f-8b3c-e49994e54870 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:43.586512 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ed86ecb4-b412-447f-8b3c-e49994e54870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:43.698645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86ecb4-b412-447f-8b3c-e49994e54870 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:43.711764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec863fec-fb5e-45d6-8ff4-7837af07287b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:43.712400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec863fec-fb5e-45d6-8ff4-7837af07287b service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:43.712680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec863fec-fb5e-45d6-8ff4-7837af07287b service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:43.713392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec863fec-fb5e-45d6-8ff4-7837af07287b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:43.740439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec863fec-fb5e-45d6-8ff4-7837af07287b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:43.743475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec863fec-fb5e-45d6-8ff4-7837af07287b service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:43.774129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 831/1658] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:44.095665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125eaaf-bc1e-415d-9ef2-22dfb11fae40 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:44.101395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-1125eaaf-bc1e-415d-9ef2-22dfb11fae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:44.123534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1125eaaf-bc1e-415d-9ef2-22dfb11fae40 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:44.135607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692aa8a-9cae-47c3-9905-85e674cdb30f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:44.136418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692aa8a-9cae-47c3-9905-85e674cdb30f service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:44.136847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692aa8a-9cae-47c3-9905-85e674cdb30f service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:44.137283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692aa8a-9cae-47c3-9905-85e674cdb30f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:44.315193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:44.319420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692aa8a-9cae-47c3-9905-85e674cdb30f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:44.320973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692aa8a-9cae-47c3-9905-85e674cdb30f service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:44.332665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 828/1659] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:48.348836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b9322-280c-4076-a2e8-4fb40217700c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:48.349353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b9322-280c-4076-a2e8-4fb40217700c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:48.352493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b9322-280c-4076-a2e8-4fb40217700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:48.453078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-475b9322-280c-4076-a2e8-4fb40217700c None None] MFA Rules not processed for user `f4ee83b28c5f417792da4ab321a174d0`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:48.454376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-475b9322-280c-4076-a2e8-4fb40217700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:48.481720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:48.670215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 832/1660] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:48.672278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608a2dd-8f9e-43b8-ab4b-f0773c65be96 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:48.673087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608a2dd-8f9e-43b8-ab4b-f0773c65be96 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:48.673476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608a2dd-8f9e-43b8-ab4b-f0773c65be96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:48.711373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dd4a0-c69d-437f-bfaf-e2d97462c1ef None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:48.711733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dd4a0-c69d-437f-bfaf-e2d97462c1ef None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:48.712012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409dd4a0-c69d-437f-bfaf-e2d97462c1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:48.737883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b608a2dd-8f9e-43b8-ab4b-f0773c65be96 None None] MFA Rules not processed for user `f4ee83b28c5f417792da4ab321a174d0`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:48.741984 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b608a2dd-8f9e-43b8-ab4b-f0773c65be96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:48.763050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 829/1661] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:48.775142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608a2dd-8f9e-43b8-ab4b-f0773c65be96 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:48.777395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b608a2dd-8f9e-43b8-ab4b-f0773c65be96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:48.803865 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-409dd4a0-c69d-437f-bfaf-e2d97462c1ef None None] MFA Rules not processed for user `84c46c4817184c9d8554c956bb7e48dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:48.805391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-409dd4a0-c69d-437f-bfaf-e2d97462c1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:48.976602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 833/1662] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:51:49.004758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dd4a0-c69d-437f-bfaf-e2d97462c1ef None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:49.006256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-409dd4a0-c69d-437f-bfaf-e2d97462c1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:49.007942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608a2dd-8f9e-43b8-ab4b-f0773c65be96 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:49.035352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ead24-a73c-45a0-9e4d-795da3fa298f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:49.035982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ead24-a73c-45a0-9e4d-795da3fa298f service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:49.036261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ead24-a73c-45a0-9e4d-795da3fa298f service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:49.036533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ead24-a73c-45a0-9e4d-795da3fa298f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:49.061345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409dd4a0-c69d-437f-bfaf-e2d97462c1ef None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:49.072541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ead24-a73c-45a0-9e4d-795da3fa298f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:49.075260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ead24-a73c-45a0-9e4d-795da3fa298f service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:49.086817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a63a45-7db9-4a3a-ae26-5e666d49f0c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:49.087459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a63a45-7db9-4a3a-ae26-5e666d49f0c6 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:49.087737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a63a45-7db9-4a3a-ae26-5e666d49f0c6 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:49.088007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a63a45-7db9-4a3a-ae26-5e666d49f0c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:49.113340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a63a45-7db9-4a3a-ae26-5e666d49f0c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:49.114992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a63a45-7db9-4a3a-ae26-5e666d49f0c6 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:49.118831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 830/1663] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:51:49.149286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 834/1664] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:51:51.998294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064db795-0896-448c-bbb7-d7eee613a4e8 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:51.998768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064db795-0896-448c-bbb7-d7eee613a4e8 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:51.998959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064db795-0896-448c-bbb7-d7eee613a4e8 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:52.001326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 831/1665] 10.222.0.79 () {54 vars in 912 bytes} [Tue Aug 9 16:51:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:51:52.018968 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ca5cf-eb27-492c-bbc1-36691c4560e0 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:52.020752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ca5cf-eb27-492c-bbc1-36691c4560e0 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:52.021187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ca5cf-eb27-492c-bbc1-36691c4560e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:52.113669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a94ca5cf-eb27-492c-bbc1-36691c4560e0 None None] MFA Rules not processed for user `c2f11dcf2c3c41a0b435a90af74c2e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:52.115211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a94ca5cf-eb27-492c-bbc1-36691c4560e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:52.256370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 835/1666] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 16:51:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:52.283633 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064db795-0896-448c-bbb7-d7eee613a4e8 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:52.285648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-064db795-0896-448c-bbb7-d7eee613a4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:52.346701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064db795-0896-448c-bbb7-d7eee613a4e8 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:52.365793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858099e-056c-40f0-91e1-2764734f1d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:52.369598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858099e-056c-40f0-91e1-2764734f1d6f service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:52.372970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858099e-056c-40f0-91e1-2764734f1d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:52.373308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858099e-056c-40f0-91e1-2764734f1d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:52.395942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f858099e-056c-40f0-91e1-2764734f1d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:52.401069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f858099e-056c-40f0-91e1-2764734f1d6f service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:52.414018 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bc218-b46e-4742-ba78-90db198c12a6 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:52.414346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bc218-b46e-4742-ba78-90db198c12a6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:52.414960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bc218-b46e-4742-ba78-90db198c12a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:52.432851 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 832/1667] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:52.464899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-f23bc218-b46e-4742-ba78-90db198c12a6 None None] MFA Rules not processed for user `be94daa21a15438f8ff005e44b133363`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:52.466335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-f23bc218-b46e-4742-ba78-90db198c12a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:52.500019 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:52.610475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 836/1668] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:52.620540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e970aa-5463-4ba5-9bba-316ce25c052c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:52.620820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e970aa-5463-4ba5-9bba-316ce25c052c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:52.621259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e970aa-5463-4ba5-9bba-316ce25c052c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:52.712473 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-23e970aa-5463-4ba5-9bba-316ce25c052c None None] MFA Rules not processed for user `be94daa21a15438f8ff005e44b133363`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:52.713917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-23e970aa-5463-4ba5-9bba-316ce25c052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:52.726170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 833/1669] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:52.732800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bc218-b46e-4742-ba78-90db198c12a6 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:52.747738 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bc218-b46e-4742-ba78-90db198c12a6 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:52.754909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef486b-1e44-4eee-9e91-d65c92380d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:52.755735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef486b-1e44-4eee-9e91-d65c92380d77 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:52.756278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef486b-1e44-4eee-9e91-d65c92380d77 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:52.756716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef486b-1e44-4eee-9e91-d65c92380d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:52.771337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef486b-1e44-4eee-9e91-d65c92380d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:52.773690 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef486b-1e44-4eee-9e91-d65c92380d77 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:52.787581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 837/1670] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:51:53.829354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a399c7-2812-4797-8e41-2208dfc1f657 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:53.829892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a399c7-2812-4797-8e41-2208dfc1f657 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:53.829892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a399c7-2812-4797-8e41-2208dfc1f657 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:53.831982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 834/1671] 10.222.0.79 () {54 vars in 912 bytes} [Tue Aug 9 16:51:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:51:53.842710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a318a-4d2f-403e-90de-000515dcbc62 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:53.845554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a318a-4d2f-403e-90de-000515dcbc62 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:53.845554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a318a-4d2f-403e-90de-000515dcbc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:53.919263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9a9a318a-4d2f-403e-90de-000515dcbc62 None None] MFA Rules not processed for user `c2f11dcf2c3c41a0b435a90af74c2e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:53.923894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9a9a318a-4d2f-403e-90de-000515dcbc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:53.956389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 838/1672] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 16:51:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:51:54.063816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a399c7-2812-4797-8e41-2208dfc1f657 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:54.082152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a399c7-2812-4797-8e41-2208dfc1f657 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:54.094595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6655d4-8ea7-4a79-a6e4-0fbb46e1bc19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:54.095583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6655d4-8ea7-4a79-a6e4-0fbb46e1bc19 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:54.095878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6655d4-8ea7-4a79-a6e4-0fbb46e1bc19 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:54.096429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6655d4-8ea7-4a79-a6e4-0fbb46e1bc19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:54.119191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6655d4-8ea7-4a79-a6e4-0fbb46e1bc19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:54.123810 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6655d4-8ea7-4a79-a6e4-0fbb46e1bc19 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:54.149474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 835/1673] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:54.204298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20209781-bc65-4bd5-9195-9224ec64ae6d None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:54.204595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20209781-bc65-4bd5-9195-9224ec64ae6d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:54.204830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20209781-bc65-4bd5-9195-9224ec64ae6d None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:54.206451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 839/1674] 10.222.0.79 () {54 vars in 961 bytes} [Tue Aug 9 16:51:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:51:54.213889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754cfe1-7068-4c9a-9e90-19b2550137c8 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:54.214205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754cfe1-7068-4c9a-9e90-19b2550137c8 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:54.215503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754cfe1-7068-4c9a-9e90-19b2550137c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:54.246875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c754cfe1-7068-4c9a-9e90-19b2550137c8 None None] MFA Rules not processed for user `d7bc938254514471801c88761525e883`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:54.250084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c754cfe1-7068-4c9a-9e90-19b2550137c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:54.396052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 836/1675] 10.222.0.79 () {58 vars in 1058 bytes} [Tue Aug 9 16:51:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:54.408854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20209781-bc65-4bd5-9195-9224ec64ae6d None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:54.431090 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20209781-bc65-4bd5-9195-9224ec64ae6d None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:54.438393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ddc4b-072f-49bf-9aa8-c275e7a03290 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:54.439601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ddc4b-072f-49bf-9aa8-c275e7a03290 service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:54.440207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ddc4b-072f-49bf-9aa8-c275e7a03290 service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:54.440624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ddc4b-072f-49bf-9aa8-c275e7a03290 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:54.442766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-458ddc4b-072f-49bf-9aa8-c275e7a03290 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:54.501269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ddc4b-072f-49bf-9aa8-c275e7a03290 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:54.506115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ddc4b-072f-49bf-9aa8-c275e7a03290 service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:54.551415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 840/1676] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:55.373861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781f2fd-b862-4c17-9fdd-6dbd0d5ba186 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:55.381691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781f2fd-b862-4c17-9fdd-6dbd0d5ba186 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:55.381691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781f2fd-b862-4c17-9fdd-6dbd0d5ba186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:55.412991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6a525-6efb-434f-b9b3-98bac6a1e003 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:55.413857 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6a525-6efb-434f-b9b3-98bac6a1e003 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:55.414302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6a525-6efb-434f-b9b3-98bac6a1e003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:55.416912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0781f2fd-b862-4c17-9fdd-6dbd0d5ba186 None None] MFA Rules not processed for user `6b7457ab864e4df6b10a195f8500c14c`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:55.422731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0781f2fd-b862-4c17-9fdd-6dbd0d5ba186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:55.512063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-04d6a525-6efb-434f-b9b3-98bac6a1e003 None None] MFA Rules not processed for user `ccad0b7ad5274d02b43f9c2fb857452a`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:55.513308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-04d6a525-6efb-434f-b9b3-98bac6a1e003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:55.544483 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:55.710537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 841/1677] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:51:55.719437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6a525-6efb-434f-b9b3-98bac6a1e003 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:55.727717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 837/1678] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:55.741065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6a525-6efb-434f-b9b3-98bac6a1e003 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:55.749906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80300f0-2554-4303-aeae-8d9c8165a379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:55.761128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80300f0-2554-4303-aeae-8d9c8165a379 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:55.761836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80300f0-2554-4303-aeae-8d9c8165a379 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:55.762511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80300f0-2554-4303-aeae-8d9c8165a379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:55.781491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80300f0-2554-4303-aeae-8d9c8165a379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:55.783512 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80300f0-2554-4303-aeae-8d9c8165a379 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:55.814388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 842/1679] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:56.020808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781f2fd-b862-4c17-9fdd-6dbd0d5ba186 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:56.062666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da8e24-56c4-4ab2-917c-c69a80d96fd3 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:56.062931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da8e24-56c4-4ab2-917c-c69a80d96fd3 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:56.063211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da8e24-56c4-4ab2-917c-c69a80d96fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:56.074870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781f2fd-b862-4c17-9fdd-6dbd0d5ba186 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:56.086663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d88e1cf-52c0-4dc1-9a88-c73c56c2eb02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:56.087322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e1cf-52c0-4dc1-9a88-c73c56c2eb02 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:56.087598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e1cf-52c0-4dc1-9a88-c73c56c2eb02 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:56.087871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88e1cf-52c0-4dc1-9a88-c73c56c2eb02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:56.103944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88e1cf-52c0-4dc1-9a88-c73c56c2eb02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:56.106390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d88e1cf-52c0-4dc1-9a88-c73c56c2eb02 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:56.125940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 838/1680] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:51:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:51:56.127574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-86da8e24-56c4-4ab2-917c-c69a80d96fd3 None None] MFA Rules not processed for user `84c46c4817184c9d8554c956bb7e48dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:56.135137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-86da8e24-56c4-4ab2-917c-c69a80d96fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:56.150249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 843/1681] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:56.438109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bb259-592c-4d65-a876-6feb5f9eb804 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:56.438902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bb259-592c-4d65-a876-6feb5f9eb804 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:56.439310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5bb259-592c-4d65-a876-6feb5f9eb804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:56.539293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-fb5bb259-592c-4d65-a876-6feb5f9eb804 None None] MFA Rules not processed for user `be94daa21a15438f8ff005e44b133363`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:51:56.544900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fb5bb259-592c-4d65-a876-6feb5f9eb804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:56.585423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 839/1682] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:51:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:51:59.402317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da8e24-56c4-4ab2-917c-c69a80d96fd3 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:51:59.405083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-86da8e24-56c4-4ab2-917c-c69a80d96fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:51:59.628384 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:51:59.672244 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da8e24-56c4-4ab2-917c-c69a80d96fd3 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:51:59.692999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:51:59.693662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:51:59.693937 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:51:59.694231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] PATH_INFO: `/v3/users/071cf79ebf6245afa2b7f0d23f9393e4` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:51:59.696410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] RBAC: Authorizing `identity:delete_user(user_id=071cf79ebf6245afa2b7f0d23f9393e4)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:51:59.698385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:51:59.954137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '071cf79ebf6245afa2b7f0d23f9393e4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:00.064465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '071cf79ebf6245afa2b7f0d23f9393e4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:00.082777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '071cf79ebf6245afa2b7f0d23f9393e4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:00.125618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '071cf79ebf6245afa2b7f0d23f9393e4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:00.126497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '071cf79ebf6245afa2b7f0d23f9393e4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:00.143314 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 844/1683] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:51:59 2022] DELETE /identity/v3/users/071cf79ebf6245afa2b7f0d23f9393e4 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:00.148948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86ba22-2287-481d-82be-42913ee70007 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:00.149493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86ba22-2287-481d-82be-42913ee70007 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:00.149568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86ba22-2287-481d-82be-42913ee70007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:00.222206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-fc86ba22-2287-481d-82be-42913ee70007 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:00.222206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fc86ba22-2287-481d-82be-42913ee70007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:00.436310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 840/1684] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:00.457468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:00.459038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:00.628049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf31bb-e381-4857-8a73-ca925c3aa21b admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:00.630825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86ba22-2287-481d-82be-42913ee70007 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:00.637374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360e93d-a7ef-4d55-aa2a-603ba3fc4629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:00.638036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fc86ba22-2287-481d-82be-42913ee70007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:00.638903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360e93d-a7ef-4d55-aa2a-603ba3fc4629 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:00.641522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360e93d-a7ef-4d55-aa2a-603ba3fc4629 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:00.654128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360e93d-a7ef-4d55-aa2a-603ba3fc4629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:00.711578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360e93d-a7ef-4d55-aa2a-603ba3fc4629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:00.715598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360e93d-a7ef-4d55-aa2a-603ba3fc4629 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:00.724396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86ba22-2287-481d-82be-42913ee70007 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:00.743041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 845/1685] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:00.745155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:00.748199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:00.752287 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:00.752876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] PATH_INFO: `/v3/users/0d46ec7a77eb419e9686cc88245ec288` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:00.758465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d46ec7a77eb419e9686cc88245ec288)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:00.762470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: WARNING py.warnings [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:52:00.762470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: warnings.warn(msg) Aug 09 16:52:00.762470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: Aug 09 16:52:00.767069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:01.035447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e58fb-cf60-4c3f-88c3-4f883c4e85d1 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:01.035739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e58fb-cf60-4c3f-88c3-4f883c4e85d1 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:01.036058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e58fb-cf60-4c3f-88c3-4f883c4e85d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:01.112691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d46ec7a77eb419e9686cc88245ec288'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:01.156617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ce3e58fb-cf60-4c3f-88c3-4f883c4e85d1 None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:01.157611 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ce3e58fb-cf60-4c3f-88c3-4f883c4e85d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:01.194000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d46ec7a77eb419e9686cc88245ec288'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:01.236193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d46ec7a77eb419e9686cc88245ec288'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:01.237032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d46ec7a77eb419e9686cc88245ec288'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:01.237711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-05bf4c35-56a3-42c7-8f4b-b37d278c4028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d46ec7a77eb419e9686cc88245ec288'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:01.275491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 841/1686] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:00 2022] DELETE /identity/v3/users/0d46ec7a77eb419e9686cc88245ec288 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:01.280051 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4196e0-e449-4913-883a-e5c22a3de1f9 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:01.280403 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4196e0-e449-4913-883a-e5c22a3de1f9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:01.280741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4196e0-e449-4913-883a-e5c22a3de1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:01.288494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:01.338720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 846/1687] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 16:52:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 09 16:52:01.369204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-5e4196e0-e449-4913-883a-e5c22a3de1f9 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:01.373681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5e4196e0-e449-4913-883a-e5c22a3de1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:01.395937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:01.503302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 842/1688] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:52:01.541765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e58fb-cf60-4c3f-88c3-4f883c4e85d1 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:01.543733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ce3e58fb-cf60-4c3f-88c3-4f883c4e85d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:01.570985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e58fb-cf60-4c3f-88c3-4f883c4e85d1 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:01.674771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7df518-6733-4373-8e9f-80197664d926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:01.675725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7df518-6733-4373-8e9f-80197664d926 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:01.676358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7df518-6733-4373-8e9f-80197664d926 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:01.676796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7df518-6733-4373-8e9f-80197664d926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:01.715355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7df518-6733-4373-8e9f-80197664d926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:01.718478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7df518-6733-4373-8e9f-80197664d926 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:01.737775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 847/1689] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:52:01.850976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4196e0-e449-4913-883a-e5c22a3de1f9 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:01.853011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5e4196e0-e449-4913-883a-e5c22a3de1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:02.043529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4196e0-e449-4913-883a-e5c22a3de1f9 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:02.050526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130271c4-876c-48ea-8b2a-eb292b3effba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:02.051228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130271c4-876c-48ea-8b2a-eb292b3effba service swift] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:02.051604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130271c4-876c-48ea-8b2a-eb292b3effba service swift] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:02.051949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130271c4-876c-48ea-8b2a-eb292b3effba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:02.228790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130271c4-876c-48ea-8b2a-eb292b3effba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:02.235122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130271c4-876c-48ea-8b2a-eb292b3effba service swift] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:02.273912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 843/1690] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 16:52:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 16:52:02.500035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecd741-3575-42fa-b3f7-298ad4c33fec None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:02.500320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecd741-3575-42fa-b3f7-298ad4c33fec None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:02.500791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ecd741-3575-42fa-b3f7-298ad4c33fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:02.563427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-89ecd741-3575-42fa-b3f7-298ad4c33fec None None] MFA Rules not processed for user `ccad0b7ad5274d02b43f9c2fb857452a`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:02.568692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-89ecd741-3575-42fa-b3f7-298ad4c33fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:02.789049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 848/1691] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:02.875841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130271c4-876c-48ea-8b2a-eb292b3effba service swift] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:02.879131 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-130271c4-876c-48ea-8b2a-eb292b3effba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:02.939264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130271c4-876c-48ea-8b2a-eb292b3effba service swift] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:02.964513 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:02.966297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:02.967162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:02.968904 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] PATH_INFO: `/v3/projects/8fa9b79fe82243618fb102355a1c8ec5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:02.971088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa9b79fe82243618fb102355a1c8ec5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:02.974402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:03.166888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:03.211460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa9b79fe82243618fb102355a1c8ec5', 'request_id': 'req-ee663196-d4d7-4a4f-83da-13407c3a9878'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:03.295692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa9b79fe82243618fb102355a1c8ec5', 'request_id': 'req-ee663196-d4d7-4a4f-83da-13407c3a9878'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:03.312957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa9b79fe82243618fb102355a1c8ec5', 'request_id': 'req-ee663196-d4d7-4a4f-83da-13407c3a9878'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:03.355834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] The token cache is being invalidate because project 8fa9b79fe82243618fb102355a1c8ec5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:52:03.356591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:03.359482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ee663196-d4d7-4a4f-83da-13407c3a9878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:03.377683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 844/1692] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:02 2022] DELETE /identity/v3/projects/8fa9b79fe82243618fb102355a1c8ec5 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:03.377847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecd741-3575-42fa-b3f7-298ad4c33fec None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:03.396942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ecd741-3575-42fa-b3f7-298ad4c33fec None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:03.609444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:03.612512 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:03.613028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:03.613434 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] PATH_INFO: `/v3/users/1bdd7ff53de24613a215a51b6cf253ca` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:03.613554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b488-d874-4b27-9deb-fe463ac86fb7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:03.613910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b488-d874-4b27-9deb-fe463ac86fb7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:03.614349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4b488-d874-4b27-9deb-fe463ac86fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:03.615431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bdd7ff53de24613a215a51b6cf253ca)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:03.620121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:03.687430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-41d4b488-d874-4b27-9deb-fe463ac86fb7 None None] MFA Rules not processed for user `c2f11dcf2c3c41a0b435a90af74c2e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:03.690475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-41d4b488-d874-4b27-9deb-fe463ac86fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:03.924553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 845/1693] 10.222.0.100 () {58 vars in 1016 bytes} [Tue Aug 9 16:52:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:04.124063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4b488-d874-4b27-9deb-fe463ac86fb7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:04.125924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-41d4b488-d874-4b27-9deb-fe463ac86fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:04.154253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4b488-d874-4b27-9deb-fe463ac86fb7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:04.179048 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:04.181360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144f3ac-2bfc-42df-8877-32b1d85117de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:04.182041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144f3ac-2bfc-42df-8877-32b1d85117de service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:04.188203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144f3ac-2bfc-42df-8877-32b1d85117de service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:04.188203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144f3ac-2bfc-42df-8877-32b1d85117de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:04.203195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bdd7ff53de24613a215a51b6cf253ca'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:04.217418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144f3ac-2bfc-42df-8877-32b1d85117de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:04.220426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144f3ac-2bfc-42df-8877-32b1d85117de service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:04.260624 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 846/1694] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:52:04.349073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bdd7ff53de24613a215a51b6cf253ca'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:04.362158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bdd7ff53de24613a215a51b6cf253ca'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:04.402574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bdd7ff53de24613a215a51b6cf253ca'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:04.404067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1dbc1528-fe5e-4ce0-afc7-96a1fb5a7b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bdd7ff53de24613a215a51b6cf253ca'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:04.428771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 849/1695] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:03 2022] DELETE /identity/v3/users/1bdd7ff53de24613a215a51b6cf253ca => generated 0 bytes in 1048 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:04.437733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144f3ac-2bfc-42df-8877-32b1d85117de service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:04.440763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0144f3ac-2bfc-42df-8877-32b1d85117de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:04.471187 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144f3ac-2bfc-42df-8877-32b1d85117de service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:04.683852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:04.684873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:04.685392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:04.685815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] PATH_INFO: `/v3/projects/8c56db670e834c97b57386cd583ca59f` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:04.692601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c56db670e834c97b57386cd583ca59f)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:04.708247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:04.958888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c56db670e834c97b57386cd583ca59f', 'request_id': 'req-fc5b3403-a76c-4491-a025-fd101f0c459a'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:05.052227 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c56db670e834c97b57386cd583ca59f', 'request_id': 'req-fc5b3403-a76c-4491-a025-fd101f0c459a'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:05.067419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c56db670e834c97b57386cd583ca59f', 'request_id': 'req-fc5b3403-a76c-4491-a025-fd101f0c459a'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:05.071200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:05.100430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] The token cache is being invalidate because project 8c56db670e834c97b57386cd583ca59f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:52:05.101050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:05.102232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fc5b3403-a76c-4491-a025-fd101f0c459a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:05.104379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 847/1696] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:04 2022] DELETE /identity/v3/projects/8c56db670e834c97b57386cd583ca59f => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:05.112686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae018d7-05b9-4a08-8c62-e71a1140f4cc None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:05.113612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae018d7-05b9-4a08-8c62-e71a1140f4cc None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:05.114869 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae018d7-05b9-4a08-8c62-e71a1140f4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:05.191830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-cae018d7-05b9-4a08-8c62-e71a1140f4cc None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:05.196330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cae018d7-05b9-4a08-8c62-e71a1140f4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:05.369043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 850/1697] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:05.381533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f569c-1b29-49e3-ba07-afa941169f7c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:05.381828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f569c-1b29-49e3-ba07-afa941169f7c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:05.382129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f569c-1b29-49e3-ba07-afa941169f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:05.420768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0e0f569c-1b29-49e3-ba07-afa941169f7c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:05.422121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0e0f569c-1b29-49e3-ba07-afa941169f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:05.442993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 848/1698] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:05.447419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae018d7-05b9-4a08-8c62-e71a1140f4cc None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:05.481718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae018d7-05b9-4a08-8c62-e71a1140f4cc None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:05.487508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee83a2-e213-46d8-8bf0-cb9c71d88bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:05.488250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee83a2-e213-46d8-8bf0-cb9c71d88bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:05.489024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee83a2-e213-46d8-8bf0-cb9c71d88bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:05.489669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee83a2-e213-46d8-8bf0-cb9c71d88bc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:05.490759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee83a2-e213-46d8-8bf0-cb9c71d88bc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:52:05.491185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee83a2-e213-46d8-8bf0-cb9c71d88bc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:05.492851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee83a2-e213-46d8-8bf0-cb9c71d88bc8 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:05.508802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 851/1699] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:52:05.513622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f569c-1b29-49e3-ba07-afa941169f7c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:05.532897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f569c-1b29-49e3-ba07-afa941169f7c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:05.540350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28289-a29e-4ff0-83f5-7f231dc69146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:05.541283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28289-a29e-4ff0-83f5-7f231dc69146 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:05.541715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28289-a29e-4ff0-83f5-7f231dc69146 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:05.542202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28289-a29e-4ff0-83f5-7f231dc69146 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:05.543030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b28289-a29e-4ff0-83f5-7f231dc69146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:05.546405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b28289-a29e-4ff0-83f5-7f231dc69146 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:05.662720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 849/1700] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:05 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:05.666985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee83a2-e213-46d8-8bf0-cb9c71d88bc8 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:05.683476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee83a2-e213-46d8-8bf0-cb9c71d88bc8 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:05.778742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793a136-df21-4ddf-be02-acf93bef0e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:05.779415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793a136-df21-4ddf-be02-acf93bef0e7d admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:05.779868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793a136-df21-4ddf-be02-acf93bef0e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:05.780282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793a136-df21-4ddf-be02-acf93bef0e7d admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:05.781142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793a136-df21-4ddf-be02-acf93bef0e7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:05.783166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793a136-df21-4ddf-be02-acf93bef0e7d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:05.919201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 852/1701] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:05 2022] POST /identity/v3/users => generated 294 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:05.926874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28289-a29e-4ff0-83f5-7f231dc69146 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:05.958946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28289-a29e-4ff0-83f5-7f231dc69146 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:05.980045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f1878-02d0-455c-b134-de3fef56e8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:05.980045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f1878-02d0-455c-b134-de3fef56e8bc admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:05.990281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f1878-02d0-455c-b134-de3fef56e8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:05.990281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f1878-02d0-455c-b134-de3fef56e8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:05.990281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f1878-02d0-455c-b134-de3fef56e8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:05.990281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f1878-02d0-455c-b134-de3fef56e8bc admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:06.024267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 850/1702] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:06.026100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793a136-df21-4ddf-be02-acf93bef0e7d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:06.070313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793a136-df21-4ddf-be02-acf93bef0e7d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:06.088436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d74b8-5d72-4660-8ec2-8997a984f19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:06.093998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d74b8-5d72-4660-8ec2-8997a984f19a admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:06.094292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d74b8-5d72-4660-8ec2-8997a984f19a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:06.094567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d74b8-5d72-4660-8ec2-8997a984f19a admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:06.095802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d74b8-5d72-4660-8ec2-8997a984f19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:06.099167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d74b8-5d72-4660-8ec2-8997a984f19a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:06.162331 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 853/1703] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:06.164774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f1878-02d0-455c-b134-de3fef56e8bc admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:06.184298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f1878-02d0-455c-b134-de3fef56e8bc admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:06.205493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0091ed3-2516-4c98-b44a-f7db331ae524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:06.210905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0091ed3-2516-4c98-b44a-f7db331ae524 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:06.211498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0091ed3-2516-4c98-b44a-f7db331ae524 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:06.213279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0091ed3-2516-4c98-b44a-f7db331ae524 admin admin] PATH_INFO: `/v3/projects/0a27a217d6874b6a8ee7eb932c2509f5/users/f0842f75784e457499df6a0273a19120/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:06.281280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0091ed3-2516-4c98-b44a-f7db331ae524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a27a217d6874b6a8ee7eb932c2509f5, user_id=f0842f75784e457499df6a0273a19120, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:06.293172 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0091ed3-2516-4c98-b44a-f7db331ae524 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:06.309924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d74b8-5d72-4660-8ec2-8997a984f19a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:06.325403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-355d74b8-5d72-4660-8ec2-8997a984f19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:06.356706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d74b8-5d72-4660-8ec2-8997a984f19a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:06.364042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:06.364833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:06.365277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:06.365986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] PATH_INFO: `/v3/projects/2f7ad8d1fa56413a894e461e35422074` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:06.368275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f7ad8d1fa56413a894e461e35422074)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:06.370082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: WARNING py.warnings [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 09 16:52:06.370082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: warnings.warn(msg) Aug 09 16:52:06.370082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: Aug 09 16:52:06.371677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:06.401181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:06.414531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 851/1704] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:06 2022] PUT /identity/v3/projects/0a27a217d6874b6a8ee7eb932c2509f5/users/f0842f75784e457499df6a0273a19120/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:06.428468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0ec91-c106-45d0-a392-eee090abe667 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:06.429023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0ec91-c106-45d0-a392-eee090abe667 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:06.429449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0ec91-c106-45d0-a392-eee090abe667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:06.521020 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f2f0ec91-c106-45d0-a392-eee090abe667 None None] MFA Rules not processed for user `f0842f75784e457499df6a0273a19120`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:06.532241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f2f0ec91-c106-45d0-a392-eee090abe667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:06.635815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f7ad8d1fa56413a894e461e35422074', 'request_id': 'req-6375d725-4a13-460a-970f-0e95cb881f4d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:06.744978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f7ad8d1fa56413a894e461e35422074', 'request_id': 'req-6375d725-4a13-460a-970f-0e95cb881f4d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:06.765735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 852/1705] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:52:06.774860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526de65f-6dab-45cb-b40f-39d673286c4a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:06.776582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526de65f-6dab-45cb-b40f-39d673286c4a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:06.777074 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526de65f-6dab-45cb-b40f-39d673286c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:06.831685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f7ad8d1fa56413a894e461e35422074', 'request_id': 'req-6375d725-4a13-460a-970f-0e95cb881f4d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:06.884487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-526de65f-6dab-45cb-b40f-39d673286c4a None None] MFA Rules not processed for user `6b7457ab864e4df6b10a195f8500c14c`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:06.887155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-526de65f-6dab-45cb-b40f-39d673286c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:06.894025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] The token cache is being invalidate because project 2f7ad8d1fa56413a894e461e35422074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:52:06.895219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:06.896128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:06.905081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 854/1706] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:06 2022] DELETE /identity/v3/projects/2f7ad8d1fa56413a894e461e35422074 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:52:06.906864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:06.909397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:07.089418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 853/1707] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:07.090382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b256cd2-ee49-46a6-929c-7550c2bc5210 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:07.093901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b256cd2-ee49-46a6-929c-7550c2bc5210 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:07.094724 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b256cd2-ee49-46a6-929c-7550c2bc5210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:07.186815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2b256cd2-ee49-46a6-929c-7550c2bc5210 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:07.193053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2b256cd2-ee49-46a6-929c-7550c2bc5210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:07.259729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:07.340986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375d725-4a13-460a-970f-0e95cb881f4d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:07.388227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424320-24d9-4580-b678-e7e197986a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:07.390087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424320-24d9-4580-b678-e7e197986a96 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:07.392556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424320-24d9-4580-b678-e7e197986a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:07.393106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424320-24d9-4580-b678-e7e197986a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:07.395921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36424320-24d9-4580-b678-e7e197986a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:07.405486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36424320-24d9-4580-b678-e7e197986a96 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:07.540038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 854/1708] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:07.555988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c62bc-3812-4d84-83c2-fb9f44db30be None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:07.556334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c62bc-3812-4d84-83c2-fb9f44db30be None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:07.556658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c62bc-3812-4d84-83c2-fb9f44db30be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:07.614548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 855/1709] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:06 2022] POST /identity/v3/projects => generated 387 bytes in 709 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:07.627827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424320-24d9-4580-b678-e7e197986a96 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:07.631522 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-3d4c62bc-3812-4d84-83c2-fb9f44db30be None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:07.633275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3d4c62bc-3812-4d84-83c2-fb9f44db30be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:07.798747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 855/1710] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:07.806001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c62bc-3812-4d84-83c2-fb9f44db30be None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:07.833049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c62bc-3812-4d84-83c2-fb9f44db30be None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:07.888617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a7344-8484-436b-b1a2-34ae0f6a574d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:07.888617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a7344-8484-436b-b1a2-34ae0f6a574d admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:07.889148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424320-24d9-4580-b678-e7e197986a96 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:07.889326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a7344-8484-436b-b1a2-34ae0f6a574d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:07.889326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a7344-8484-436b-b1a2-34ae0f6a574d admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:07.889591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a7344-8484-436b-b1a2-34ae0f6a574d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:52:07.889743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a7344-8484-436b-b1a2-34ae0f6a574d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:07.891373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a7344-8484-436b-b1a2-34ae0f6a574d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:07.915959 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f414ee5-c659-4473-9e0d-558408adfd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:07.917029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f414ee5-c659-4473-9e0d-558408adfd7f admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:07.917426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f414ee5-c659-4473-9e0d-558408adfd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:07.917915 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f414ee5-c659-4473-9e0d-558408adfd7f admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:07.925724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f414ee5-c659-4473-9e0d-558408adfd7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:07.931457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f414ee5-c659-4473-9e0d-558408adfd7f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:07.945510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 856/1711] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:52:07.948501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a7344-8484-436b-b1a2-34ae0f6a574d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:07.978237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a7344-8484-436b-b1a2-34ae0f6a574d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:08.014578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ae0c4-6989-427e-beb1-d1dd1b4528c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:08.024095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ae0c4-6989-427e-beb1-d1dd1b4528c5 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:08.028257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ae0c4-6989-427e-beb1-d1dd1b4528c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:08.029582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ae0c4-6989-427e-beb1-d1dd1b4528c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:08.030976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ae0c4-6989-427e-beb1-d1dd1b4528c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:08.043834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ae0c4-6989-427e-beb1-d1dd1b4528c5 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:08.128341 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 856/1712] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:07 2022] POST /identity/v3/users => generated 294 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:08.136692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f414ee5-c659-4473-9e0d-558408adfd7f admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:08.203264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f414ee5-c659-4473-9e0d-558408adfd7f admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:08.218902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 857/1713] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:07 2022] POST /identity/v3/projects => generated 365 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:08.220056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ae0c4-6989-427e-beb1-d1dd1b4528c5 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:08.276424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:08.412829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ae0c4-6989-427e-beb1-d1dd1b4528c5 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:08.425677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5334787-978d-43c6-a69d-e4ddcdd7fa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:08.431418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5334787-978d-43c6-a69d-e4ddcdd7fa83 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:08.431418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5334787-978d-43c6-a69d-e4ddcdd7fa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:08.431418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5334787-978d-43c6-a69d-e4ddcdd7fa83 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:08.431418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5334787-978d-43c6-a69d-e4ddcdd7fa83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:08.437161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5334787-978d-43c6-a69d-e4ddcdd7fa83 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:08.473897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fafb01-94d1-4d3c-8955-f766aafb4ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:08.488297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fafb01-94d1-4d3c-8955-f766aafb4ffd admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:08.488984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fafb01-94d1-4d3c-8955-f766aafb4ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:08.494466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fafb01-94d1-4d3c-8955-f766aafb4ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:08.500827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fafb01-94d1-4d3c-8955-f766aafb4ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:08.535830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fafb01-94d1-4d3c-8955-f766aafb4ffd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:08.571495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 857/1714] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:08.573344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac80db-6a50-41e4-9246-5af7b06406f2 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:08.573526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac80db-6a50-41e4-9246-5af7b06406f2 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:08.573801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac80db-6a50-41e4-9246-5af7b06406f2 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:08.582750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 858/1715] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 16:52:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:52:08.594064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 858/1716] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:08 2022] POST /identity/v3/users => generated 283 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:08.595129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5334787-978d-43c6-a69d-e4ddcdd7fa83 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:08.601592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac80db-6a50-41e4-9246-5af7b06406f2 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:08.617940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5334787-978d-43c6-a69d-e4ddcdd7fa83 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:08.624012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0604-ac41-4060-a4e1-a7e9075fe35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:08.624848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0604-ac41-4060-a4e1-a7e9075fe35f admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:08.625144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0604-ac41-4060-a4e1-a7e9075fe35f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:08.625412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0604-ac41-4060-a4e1-a7e9075fe35f admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:08.626186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b0604-ac41-4060-a4e1-a7e9075fe35f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:08.629141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b0604-ac41-4060-a4e1-a7e9075fe35f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:08.665431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac80db-6a50-41e4-9246-5af7b06406f2 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:08.670756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 859/1717] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:08.672193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0604-ac41-4060-a4e1-a7e9075fe35f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:08.680627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:08.684375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217814ee-a36b-49f8-99c2-3f8d1736886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:08.686908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217814ee-a36b-49f8-99c2-3f8d1736886f admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:08.699614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0604-ac41-4060-a4e1-a7e9075fe35f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:08.702403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217814ee-a36b-49f8-99c2-3f8d1736886f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:08.703382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217814ee-a36b-49f8-99c2-3f8d1736886f admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:08.707038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217814ee-a36b-49f8-99c2-3f8d1736886f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:08.712753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217814ee-a36b-49f8-99c2-3f8d1736886f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:08.714737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa86505-512c-42cd-8045-e6383ad29b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:08.714737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa86505-512c-42cd-8045-e6383ad29b9d admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:08.714737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa86505-512c-42cd-8045-e6383ad29b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:08.733702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 859/1718] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:08.733924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa86505-512c-42cd-8045-e6383ad29b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:08.734940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa86505-512c-42cd-8045-e6383ad29b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:08.736871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa86505-512c-42cd-8045-e6383ad29b9d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:08.738099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217814ee-a36b-49f8-99c2-3f8d1736886f admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:08.751540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217814ee-a36b-49f8-99c2-3f8d1736886f admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:08.777277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd8ee7-34d5-45bc-b2a4-0883bec5b689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:08.778203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd8ee7-34d5-45bc-b2a4-0883bec5b689 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:08.778663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd8ee7-34d5-45bc-b2a4-0883bec5b689 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:08.779091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd8ee7-34d5-45bc-b2a4-0883bec5b689 admin admin] PATH_INFO: `/v3/projects/d60eafc840bf413cb0e70d2ed34a7256/users/dde11155099e4733978b458841ffff7b/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:08.801162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 860/1719] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:08.803358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa86505-512c-42cd-8045-e6383ad29b9d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:08.828061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd8ee7-34d5-45bc-b2a4-0883bec5b689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d60eafc840bf413cb0e70d2ed34a7256, user_id=dde11155099e4733978b458841ffff7b, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:08.832666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd8ee7-34d5-45bc-b2a4-0883bec5b689 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:08.852432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa86505-512c-42cd-8045-e6383ad29b9d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:08.865324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f4ac7a-87f8-41f5-99ed-0851cac20de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:08.866353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ac7a-87f8-41f5-99ed-0851cac20de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:08.874010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ac7a-87f8-41f5-99ed-0851cac20de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:08.874010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4ac7a-87f8-41f5-99ed-0851cac20de8 admin admin] PATH_INFO: `/v3/projects/0b1064c0e33e49899c6dea2b062221b3/users/6d2ef50d565a4da68cb1db947b36b5d0/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:08.908034 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4ac7a-87f8-41f5-99ed-0851cac20de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b1064c0e33e49899c6dea2b062221b3, user_id=6d2ef50d565a4da68cb1db947b36b5d0, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:08.913937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f4ac7a-87f8-41f5-99ed-0851cac20de8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:08.945063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 860/1720] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:08 2022] PUT /identity/v3/projects/d60eafc840bf413cb0e70d2ed34a7256/users/dde11155099e4733978b458841ffff7b/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:08.946784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e488d1-6833-4e49-b10e-ebf050228beb None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:08.947079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e488d1-6833-4e49-b10e-ebf050228beb None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:08.947912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e488d1-6833-4e49-b10e-ebf050228beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:09.004391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-45e488d1-6833-4e49-b10e-ebf050228beb None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:09.005749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-45e488d1-6833-4e49-b10e-ebf050228beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:09.022405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 861/1721] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:08 2022] PUT /identity/v3/projects/0b1064c0e33e49899c6dea2b062221b3/users/6d2ef50d565a4da68cb1db947b36b5d0/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:09.024103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28fbc9-91fd-4480-8714-6ece4a3e68d2 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:09.024395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28fbc9-91fd-4480-8714-6ece4a3e68d2 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:09.024676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28fbc9-91fd-4480-8714-6ece4a3e68d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:09.105193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-bf28fbc9-91fd-4480-8714-6ece4a3e68d2 None None] MFA Rules not processed for user `dde11155099e4733978b458841ffff7b`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:09.106562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bf28fbc9-91fd-4480-8714-6ece4a3e68d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:09.236757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 861/1722] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 16:52:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:09.240456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63941133-179c-40ae-8e19-e7f12417c9cb None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:09.240456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63941133-179c-40ae-8e19-e7f12417c9cb None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:09.240709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63941133-179c-40ae-8e19-e7f12417c9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:09.327691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-63941133-179c-40ae-8e19-e7f12417c9cb None None] MFA Rules not processed for user `6d2ef50d565a4da68cb1db947b36b5d0`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:09.329693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-63941133-179c-40ae-8e19-e7f12417c9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:09.348148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 862/1723] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:09.384950 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28fbc9-91fd-4480-8714-6ece4a3e68d2 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:09.564904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:09.586189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 862/1724] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:09.590426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63941133-179c-40ae-8e19-e7f12417c9cb None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:09.680082 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28fbc9-91fd-4480-8714-6ece4a3e68d2 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:09.692096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9016d-2b03-483f-96f8-5b50de124a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:09.692986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9016d-2b03-483f-96f8-5b50de124a8f admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:09.693330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9016d-2b03-483f-96f8-5b50de124a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:09.693671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9016d-2b03-483f-96f8-5b50de124a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:09.694412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9016d-2b03-483f-96f8-5b50de124a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:09.697326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9016d-2b03-483f-96f8-5b50de124a8f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:09.708720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63941133-179c-40ae-8e19-e7f12417c9cb None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:09.715608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d420fd-1afd-4860-83cd-2e42353faebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:09.717040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d420fd-1afd-4860-83cd-2e42353faebe admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:09.717512 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d420fd-1afd-4860-83cd-2e42353faebe admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:09.718154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d420fd-1afd-4860-83cd-2e42353faebe admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:09.738791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d420fd-1afd-4860-83cd-2e42353faebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:09.742597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d420fd-1afd-4860-83cd-2e42353faebe admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:09.840107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 863/1725] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:09 2022] POST /identity/v3/projects => generated 387 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:09.847335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9016d-2b03-483f-96f8-5b50de124a8f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:09.864905 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 863/1726] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:09 2022] POST /identity/v3/projects => generated 365 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:09.869088 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d420fd-1afd-4860-83cd-2e42353faebe admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:09.969356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:10.000816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9016d-2b03-483f-96f8-5b50de124a8f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:10.130609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6e952-ec9a-482e-9b43-dfd555fd6a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:10.131472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6e952-ec9a-482e-9b43-dfd555fd6a68 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:10.131602 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6e952-ec9a-482e-9b43-dfd555fd6a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:10.131907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6e952-ec9a-482e-9b43-dfd555fd6a68 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:10.134596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d420fd-1afd-4860-83cd-2e42353faebe admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:10.138376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6e952-ec9a-482e-9b43-dfd555fd6a68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:10.140570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6e952-ec9a-482e-9b43-dfd555fd6a68 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:10.143352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca071dbf-27da-4c82-8221-816405606f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:10.146141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca071dbf-27da-4c82-8221-816405606f1e admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:10.146141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca071dbf-27da-4c82-8221-816405606f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:10.146141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca071dbf-27da-4c82-8221-816405606f1e admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:10.146724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca071dbf-27da-4c82-8221-816405606f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:10.149328 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca071dbf-27da-4c82-8221-816405606f1e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:10.286370 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 864/1727] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:09 2022] POST /identity/v3/users => generated 300 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:52:10.292666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6e952-ec9a-482e-9b43-dfd555fd6a68 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:10.293005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 864/1728] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:09 2022] POST /identity/v3/users => generated 289 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:10.297858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca071dbf-27da-4c82-8221-816405606f1e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:10.311809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6e952-ec9a-482e-9b43-dfd555fd6a68 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:10.344716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca071dbf-27da-4c82-8221-816405606f1e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:10.347333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df258c-cd41-4c5d-aab8-65a15d5c9224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:10.350261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df258c-cd41-4c5d-aab8-65a15d5c9224 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:10.350580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df258c-cd41-4c5d-aab8-65a15d5c9224 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:10.352220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df258c-cd41-4c5d-aab8-65a15d5c9224 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:10.355894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df258c-cd41-4c5d-aab8-65a15d5c9224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:10.358669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df258c-cd41-4c5d-aab8-65a15d5c9224 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:10.368745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f48936-0557-48be-bfc7-8c082dc435a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:10.371241 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f48936-0557-48be-bfc7-8c082dc435a8 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:10.373675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f48936-0557-48be-bfc7-8c082dc435a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:10.374035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f48936-0557-48be-bfc7-8c082dc435a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:10.374752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 865/1729] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:52:10.378845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f48936-0557-48be-bfc7-8c082dc435a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:10.378845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f48936-0557-48be-bfc7-8c082dc435a8 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:10.386447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df258c-cd41-4c5d-aab8-65a15d5c9224 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:10.429122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df258c-cd41-4c5d-aab8-65a15d5c9224 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:10.430532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 865/1730] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:52:10.436787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f48936-0557-48be-bfc7-8c082dc435a8 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:10.442581 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316ed4b-a5a7-4c83-989c-3cdf3c1ddd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:10.443873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316ed4b-a5a7-4c83-989c-3cdf3c1ddd47 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:10.446063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316ed4b-a5a7-4c83-989c-3cdf3c1ddd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:10.446734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316ed4b-a5a7-4c83-989c-3cdf3c1ddd47 admin admin] PATH_INFO: `/v3/projects/2a4d2b10cd0e4f5da750a2b42ab841f5/users/4ee0365fa573429cba83d1ee2782bfff/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:10.502485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f48936-0557-48be-bfc7-8c082dc435a8 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:10.544293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb1d64-455d-4bc2-853e-e812f29faaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:10.545006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb1d64-455d-4bc2-853e-e812f29faaff admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:10.545357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb1d64-455d-4bc2-853e-e812f29faaff admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:10.545579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb1d64-455d-4bc2-853e-e812f29faaff admin admin] PATH_INFO: `/v3/projects/230d8d92fcb449e98770011791077bc4/users/67a235e0c02b46459d050956679c7fe7/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:10.546181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316ed4b-a5a7-4c83-989c-3cdf3c1ddd47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4d2b10cd0e4f5da750a2b42ab841f5, user_id=4ee0365fa573429cba83d1ee2782bfff, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:10.550132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316ed4b-a5a7-4c83-989c-3cdf3c1ddd47 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:10.590922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb1d64-455d-4bc2-853e-e812f29faaff admin admin] RBAC: Authorizing `identity:create_grant(project_id=230d8d92fcb449e98770011791077bc4, user_id=67a235e0c02b46459d050956679c7fe7, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:10.602078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb1d64-455d-4bc2-853e-e812f29faaff admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:10.621995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 866/1731] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:10 2022] PUT /identity/v3/projects/2a4d2b10cd0e4f5da750a2b42ab841f5/users/4ee0365fa573429cba83d1ee2782bfff/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:52:10.635573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316ed4b-a5a7-4c83-989c-3cdf3c1ddd47 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:10.712339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 866/1732] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:10 2022] PUT /identity/v3/projects/230d8d92fcb449e98770011791077bc4/users/67a235e0c02b46459d050956679c7fe7/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:10.714933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb1d64-455d-4bc2-853e-e812f29faaff admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:10.865091 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316ed4b-a5a7-4c83-989c-3cdf3c1ddd47 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:10.914533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:11.006261 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb1d64-455d-4bc2-853e-e812f29faaff admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:11.019053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a994851-b050-48de-88ce-69d1fde08ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:11.019954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994851-b050-48de-88ce-69d1fde08ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:11.020606 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994851-b050-48de-88ce-69d1fde08ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:11.020946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a994851-b050-48de-88ce-69d1fde08ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:11.021851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a994851-b050-48de-88ce-69d1fde08ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:11.024868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a994851-b050-48de-88ce-69d1fde08ec0 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:11.046974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0c7c0-f0fa-481a-b4ba-712adcdaed1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:11.050655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0c7c0-f0fa-481a-b4ba-712adcdaed1e admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:11.051352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0c7c0-f0fa-481a-b4ba-712adcdaed1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:11.051352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0c7c0-f0fa-481a-b4ba-712adcdaed1e admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:11.053170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0c7c0-f0fa-481a-b4ba-712adcdaed1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:11.053401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 867/1733] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:11.055975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0c7c0-f0fa-481a-b4ba-712adcdaed1e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:11.072100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a994851-b050-48de-88ce-69d1fde08ec0 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:11.074584 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0a994851-b050-48de-88ce-69d1fde08ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:11.075254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:11.114290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 867/1734] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 487 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:52:11.115067 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0c7c0-f0fa-481a-b4ba-712adcdaed1e admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:11.143482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0c7c0-f0fa-481a-b4ba-712adcdaed1e admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:11.157239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16835c5-9151-4a6a-9d83-641d6e049a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:11.158709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16835c5-9151-4a6a-9d83-641d6e049a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:11.158976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16835c5-9151-4a6a-9d83-641d6e049a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:11.159239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16835c5-9151-4a6a-9d83-641d6e049a7d admin admin] PATH_INFO: `/v3/projects/230d8d92fcb449e98770011791077bc4/users/67a235e0c02b46459d050956679c7fe7/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:11.166453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16835c5-9151-4a6a-9d83-641d6e049a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=230d8d92fcb449e98770011791077bc4, user_id=67a235e0c02b46459d050956679c7fe7, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:11.169435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16835c5-9151-4a6a-9d83-641d6e049a7d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:11.216652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 868/1735] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:11 2022] PUT /identity/v3/projects/230d8d92fcb449e98770011791077bc4/users/67a235e0c02b46459d050956679c7fe7/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:11.217573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16835c5-9151-4a6a-9d83-641d6e049a7d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:11.279683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a994851-b050-48de-88ce-69d1fde08ec0 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:11.520994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2ab70-8b25-4435-b0a0-b0fb9e95f727 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:11.523423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16835c5-9151-4a6a-9d83-641d6e049a7d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:11.525490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2ab70-8b25-4435-b0a0-b0fb9e95f727 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:11.551696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2ab70-8b25-4435-b0a0-b0fb9e95f727 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:11.551696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2ab70-8b25-4435-b0a0-b0fb9e95f727 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:11.551832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbc2fa-450c-4053-8c02-37c67c25d767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:11.551832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbc2fa-450c-4053-8c02-37c67c25d767 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:11.551832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbc2fa-450c-4053-8c02-37c67c25d767 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:11.551832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbc2fa-450c-4053-8c02-37c67c25d767 admin admin] PATH_INFO: `/v3/projects/2a4d2b10cd0e4f5da750a2b42ab841f5/users/4ee0365fa573429cba83d1ee2782bfff/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:11.557285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbc2fa-450c-4053-8c02-37c67c25d767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a4d2b10cd0e4f5da750a2b42ab841f5, user_id=4ee0365fa573429cba83d1ee2782bfff, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:11.563096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbc2fa-450c-4053-8c02-37c67c25d767 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:11.574779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2ab70-8b25-4435-b0a0-b0fb9e95f727 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:11.580129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2ab70-8b25-4435-b0a0-b0fb9e95f727 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:11.585975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 869/1736] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:11 2022] PUT /identity/v3/projects/2a4d2b10cd0e4f5da750a2b42ab841f5/users/4ee0365fa573429cba83d1ee2782bfff/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:11.585975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cbdb6-f60a-4481-83c7-e45d7a7440c7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:11.599324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 868/1737] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:11.609170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cbdb6-f60a-4481-83c7-e45d7a7440c7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:11.609554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cbdb6-f60a-4481-83c7-e45d7a7440c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:11.613330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864bba4-b442-47a7-98d3-aaf7e92478e0 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:11.614705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864bba4-b442-47a7-98d3-aaf7e92478e0 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:11.614705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864bba4-b442-47a7-98d3-aaf7e92478e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:11.681855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-7864bba4-b442-47a7-98d3-aaf7e92478e0 None None] MFA Rules not processed for user `4ee0365fa573429cba83d1ee2782bfff`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:11.681855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7864bba4-b442-47a7-98d3-aaf7e92478e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:11.707648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b45cbdb6-f60a-4481-83c7-e45d7a7440c7 None None] MFA Rules not processed for user `67a235e0c02b46459d050956679c7fe7`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:11.710955 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b45cbdb6-f60a-4481-83c7-e45d7a7440c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:11.882994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 870/1738] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:11.886006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 869/1739] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:11.898608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cbdb6-f60a-4481-83c7-e45d7a7440c7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:11.898608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b45cbdb6-f60a-4481-83c7-e45d7a7440c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:11.904147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864bba4-b442-47a7-98d3-aaf7e92478e0 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:11.907916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7864bba4-b442-47a7-98d3-aaf7e92478e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:12.073481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864bba4-b442-47a7-98d3-aaf7e92478e0 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:12.083166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c8de-8e6d-4d41-832f-b096e14ae95a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:12.083971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c8de-8e6d-4d41-832f-b096e14ae95a service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:12.085531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c8de-8e6d-4d41-832f-b096e14ae95a service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:12.086092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cbdb6-f60a-4481-83c7-e45d7a7440c7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:12.086422 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c8de-8e6d-4d41-832f-b096e14ae95a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:12.096456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:12.104293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d58aff-c54c-45f1-b832-d0511438515c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:12.105082 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d58aff-c54c-45f1-b832-d0511438515c service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:12.105360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d58aff-c54c-45f1-b832-d0511438515c service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:12.109726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d58aff-c54c-45f1-b832-d0511438515c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:12.116812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829c8de-8e6d-4d41-832f-b096e14ae95a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:12.120099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829c8de-8e6d-4d41-832f-b096e14ae95a service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:12.142674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d58aff-c54c-45f1-b832-d0511438515c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:12.146631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d58aff-c54c-45f1-b832-d0511438515c service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:12.148185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 870/1740] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:52:12.161415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:12.177661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 871/1741] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:14.035874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c8de-8e6d-4d41-832f-b096e14ae95a service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:14.216360 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c8de-8e6d-4d41-832f-b096e14ae95a service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:14.233039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:14.233039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:14.233039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:14.233039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] PATH_INFO: `/v3/users/6d2ef50d565a4da68cb1db947b36b5d0` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:14.233324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2ef50d565a4da68cb1db947b36b5d0)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:14.233680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:14.326424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d58aff-c54c-45f1-b832-d0511438515c service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:14.338067 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d58aff-c54c-45f1-b832-d0511438515c service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:14.343987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8030ff-0f6f-4956-8704-4c2d6973977a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:14.345560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8030ff-0f6f-4956-8704-4c2d6973977a service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:14.346050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8030ff-0f6f-4956-8704-4c2d6973977a service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:14.349003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8030ff-0f6f-4956-8704-4c2d6973977a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:14.352048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3b8030ff-0f6f-4956-8704-4c2d6973977a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:14.448139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2ef50d565a4da68cb1db947b36b5d0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:14.570924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8030ff-0f6f-4956-8704-4c2d6973977a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:14.572391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8030ff-0f6f-4956-8704-4c2d6973977a service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:14.588968 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2ef50d565a4da68cb1db947b36b5d0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:14.605003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 872/1742] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:52:14.620165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2ef50d565a4da68cb1db947b36b5d0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:14.642054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2ef50d565a4da68cb1db947b36b5d0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:14.650468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2ef50d565a4da68cb1db947b36b5d0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:14.651788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 871/1743] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:14 2022] DELETE /identity/v3/users/6d2ef50d565a4da68cb1db947b36b5d0 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:14.656808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1fda0-3a24-4f3e-a82e-99b451b073e7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:14.657110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1fda0-3a24-4f3e-a82e-99b451b073e7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:14.657382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1fda0-3a24-4f3e-a82e-99b451b073e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:14.713030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-42f1fda0-3a24-4f3e-a82e-99b451b073e7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:14.714423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-42f1fda0-3a24-4f3e-a82e-99b451b073e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:14.907786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 873/1744] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:14.926697 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:14.929575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:14.947356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:15.059993 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a729f-b182-4c37-b461-6718d4dae19d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:15.064597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a73144-4bf5-4df0-8706-6dc46bd41a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:15.069881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a73144-4bf5-4df0-8706-6dc46bd41a20 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:15.069881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a73144-4bf5-4df0-8706-6dc46bd41a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:15.069881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a73144-4bf5-4df0-8706-6dc46bd41a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:15.081696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a73144-4bf5-4df0-8706-6dc46bd41a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:15.085924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a73144-4bf5-4df0-8706-6dc46bd41a20 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:15.101488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 872/1745] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:15.371933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569089a2-337a-44e2-962a-4aba41692822 None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:15.372511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569089a2-337a-44e2-962a-4aba41692822 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:15.373141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569089a2-337a-44e2-962a-4aba41692822 None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:15.374776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 874/1746] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 16:52:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:52:15.517046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af677e-08ed-49a5-ac36-c919c59db938 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:15.517579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af677e-08ed-49a5-ac36-c919c59db938 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:15.518052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af677e-08ed-49a5-ac36-c919c59db938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:15.555598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-16af677e-08ed-49a5-ac36-c919c59db938 None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:15.556999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-16af677e-08ed-49a5-ac36-c919c59db938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:15.696224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 873/1747] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 16:52:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:52:16.264146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569089a2-337a-44e2-962a-4aba41692822 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:16.267002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-569089a2-337a-44e2-962a-4aba41692822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:16.312057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569089a2-337a-44e2-962a-4aba41692822 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:16.320924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd52d9-574c-4e38-acac-4f34a5ae23ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:16.322105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd52d9-574c-4e38-acac-4f34a5ae23ed service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:16.322798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd52d9-574c-4e38-acac-4f34a5ae23ed service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:16.323259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd52d9-574c-4e38-acac-4f34a5ae23ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:16.344273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd52d9-574c-4e38-acac-4f34a5ae23ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:16.347642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd52d9-574c-4e38-acac-4f34a5ae23ed service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:16.381643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 875/1748] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:16.766306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af677e-08ed-49a5-ac36-c919c59db938 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:16.768097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-16af677e-08ed-49a5-ac36-c919c59db938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:16.785781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af677e-08ed-49a5-ac36-c919c59db938 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:16.793095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:16.794272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:16.795431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:16.795848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] PATH_INFO: `/v3/projects/0b1064c0e33e49899c6dea2b062221b3` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:16.797660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b1064c0e33e49899c6dea2b062221b3)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:16.800226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:16.992716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b1064c0e33e49899c6dea2b062221b3', 'request_id': 'req-d4b87f76-ca41-425b-9a0e-52aee3e0b111'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:17.125491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b1064c0e33e49899c6dea2b062221b3', 'request_id': 'req-d4b87f76-ca41-425b-9a0e-52aee3e0b111'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:17.140254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b1064c0e33e49899c6dea2b062221b3', 'request_id': 'req-d4b87f76-ca41-425b-9a0e-52aee3e0b111'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:17.196080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] The token cache is being invalidate because project 0b1064c0e33e49899c6dea2b062221b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:52:17.196658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:17.205549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:17.217733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 874/1749] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:16 2022] DELETE /identity/v3/projects/0b1064c0e33e49899c6dea2b062221b3 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:17.218413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd52d9-574c-4e38-acac-4f34a5ae23ed service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:17.231478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd52d9-574c-4e38-acac-4f34a5ae23ed service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:17.335197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:17.336841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:17.337264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:17.337702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] PATH_INFO: `/v3/users/67a235e0c02b46459d050956679c7fe7` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:17.339737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67a235e0c02b46459d050956679c7fe7)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:17.341885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:17.525578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:17.719641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67a235e0c02b46459d050956679c7fe7'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:17.835219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67a235e0c02b46459d050956679c7fe7'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:17.865933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67a235e0c02b46459d050956679c7fe7'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:17.866937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67a235e0c02b46459d050956679c7fe7'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:17.868006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d31dfb7c-ee7d-4146-8412-333c255e16c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67a235e0c02b46459d050956679c7fe7'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:17.898804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 876/1750] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:17 2022] DELETE /identity/v3/users/67a235e0c02b46459d050956679c7fe7 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:52:18.851570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:18.853233 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:18.866184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b87f76-ca41-425b-9a0e-52aee3e0b111 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:18.885613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:18.928767 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:18.929474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:18.929875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:18.930254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] PATH_INFO: `/v3/projects/230d8d92fcb449e98770011791077bc4` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:18.931493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=230d8d92fcb449e98770011791077bc4)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:18.933644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:19.034980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '230d8d92fcb449e98770011791077bc4', 'request_id': 'req-afbf8458-250e-4308-8b4c-654c30d270f7'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:19.118249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '230d8d92fcb449e98770011791077bc4', 'request_id': 'req-afbf8458-250e-4308-8b4c-654c30d270f7'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:19.128091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '230d8d92fcb449e98770011791077bc4', 'request_id': 'req-afbf8458-250e-4308-8b4c-654c30d270f7'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:19.140286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] The token cache is being invalidate because project 230d8d92fcb449e98770011791077bc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:52:19.140779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:19.141485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-afbf8458-250e-4308-8b4c-654c30d270f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:19.143179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 875/1751] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:18 2022] DELETE /identity/v3/projects/230d8d92fcb449e98770011791077bc4 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:19.150014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdfdef-7946-44d4-935e-2405f33d9013 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:19.151585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdfdef-7946-44d4-935e-2405f33d9013 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:19.152196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdfdef-7946-44d4-935e-2405f33d9013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:19.176084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-8afdfdef-7946-44d4-935e-2405f33d9013 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:19.177344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8afdfdef-7946-44d4-935e-2405f33d9013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:19.253462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 877/1752] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:19.263525 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4183-a0dc-413e-98f6-c27c305c07ce None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:19.263839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4183-a0dc-413e-98f6-c27c305c07ce None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:19.264107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f4183-a0dc-413e-98f6-c27c305c07ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:19.284490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ef1f4183-a0dc-413e-98f6-c27c305c07ce None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:19.285736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ef1f4183-a0dc-413e-98f6-c27c305c07ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:19.294862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 876/1753] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:19.299061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdfdef-7946-44d4-935e-2405f33d9013 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:19.308843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afdfdef-7946-44d4-935e-2405f33d9013 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:19.314035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a310d-fe7d-4f5e-b564-9de3516ed533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:19.314631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a310d-fe7d-4f5e-b564-9de3516ed533 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:19.314903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a310d-fe7d-4f5e-b564-9de3516ed533 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:19.315166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a310d-fe7d-4f5e-b564-9de3516ed533 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:19.315806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a310d-fe7d-4f5e-b564-9de3516ed533 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:52:19.316098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a310d-fe7d-4f5e-b564-9de3516ed533 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:19.317729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62a310d-fe7d-4f5e-b564-9de3516ed533 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:19.331151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 878/1754] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:52:19.335701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f4183-a0dc-413e-98f6-c27c305c07ce None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:19.349544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f4183-a0dc-413e-98f6-c27c305c07ce None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:19.351840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04823cad-9f81-465e-a62e-e6007c6fd41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:19.352525 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04823cad-9f81-465e-a62e-e6007c6fd41b admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:19.352803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04823cad-9f81-465e-a62e-e6007c6fd41b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:19.353093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04823cad-9f81-465e-a62e-e6007c6fd41b admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:19.353799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04823cad-9f81-465e-a62e-e6007c6fd41b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:19.355749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04823cad-9f81-465e-a62e-e6007c6fd41b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:19.478125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 877/1755] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:19 2022] POST /identity/v3/projects => generated 407 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:19.479306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a310d-fe7d-4f5e-b564-9de3516ed533 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:19.490393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a310d-fe7d-4f5e-b564-9de3516ed533 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:19.583977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110edcb-661f-440e-b18c-e89ff0915e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:19.585021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110edcb-661f-440e-b18c-e89ff0915e7e admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:19.585368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110edcb-661f-440e-b18c-e89ff0915e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:19.585687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5110edcb-661f-440e-b18c-e89ff0915e7e admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:19.586440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110edcb-661f-440e-b18c-e89ff0915e7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:19.588570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5110edcb-661f-440e-b18c-e89ff0915e7e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:19.724817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 879/1756] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:19 2022] POST /identity/v3/users => generated 304 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:19.724912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04823cad-9f81-465e-a62e-e6007c6fd41b admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:19.730135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04823cad-9f81-465e-a62e-e6007c6fd41b admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:19.736219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e15332-a30e-4ede-a40d-109214328305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:19.743723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e15332-a30e-4ede-a40d-109214328305 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:19.743723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e15332-a30e-4ede-a40d-109214328305 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:19.743723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e15332-a30e-4ede-a40d-109214328305 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:19.743723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e15332-a30e-4ede-a40d-109214328305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:19.743723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e15332-a30e-4ede-a40d-109214328305 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:19.752622 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 878/1757] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:19.757026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110edcb-661f-440e-b18c-e89ff0915e7e admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:19.782354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5110edcb-661f-440e-b18c-e89ff0915e7e admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:19.800970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7adff-2c87-4f6c-9053-9c66c35f87f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:19.801605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7adff-2c87-4f6c-9053-9c66c35f87f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:19.802625 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7adff-2c87-4f6c-9053-9c66c35f87f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:19.802625 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7adff-2c87-4f6c-9053-9c66c35f87f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:19.803162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7adff-2c87-4f6c-9053-9c66c35f87f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:19.804752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7adff-2c87-4f6c-9053-9c66c35f87f4 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:19.820048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 880/1758] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:19.822312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e15332-a30e-4ede-a40d-109214328305 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:19.834903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e15332-a30e-4ede-a40d-109214328305 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:19.843299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8c2d2-716f-4099-acdf-298d603aa848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:19.844313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8c2d2-716f-4099-acdf-298d603aa848 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:19.844805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8c2d2-716f-4099-acdf-298d603aa848 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:19.845286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8c2d2-716f-4099-acdf-298d603aa848 admin admin] PATH_INFO: `/v3/projects/37616e70da8f46afa47a6e367c82d459/users/969f8bc72e3e4295b90eb7eb0cffa50e/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:19.873301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8c2d2-716f-4099-acdf-298d603aa848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37616e70da8f46afa47a6e367c82d459, user_id=969f8bc72e3e4295b90eb7eb0cffa50e, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:19.876845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8c2d2-716f-4099-acdf-298d603aa848 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:19.971488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 879/1759] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:19 2022] PUT /identity/v3/projects/37616e70da8f46afa47a6e367c82d459/users/969f8bc72e3e4295b90eb7eb0cffa50e/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:19.983566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f53422-93d0-4bd0-9592-a7002f7f0747 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:19.983846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f53422-93d0-4bd0-9592-a7002f7f0747 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:19.984115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f53422-93d0-4bd0-9592-a7002f7f0747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:20.007066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:20.020605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d0f53422-93d0-4bd0-9592-a7002f7f0747 None None] MFA Rules not processed for user `969f8bc72e3e4295b90eb7eb0cffa50e`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:20.022042 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d0f53422-93d0-4bd0-9592-a7002f7f0747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:20.112379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 881/1760] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:19 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:20.116990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8c2d2-716f-4099-acdf-298d603aa848 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:20.177538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:20.195220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8c2d2-716f-4099-acdf-298d603aa848 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:20.200229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01c79d-2e90-4089-a932-9b52db38fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:20.201557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01c79d-2e90-4089-a932-9b52db38fc78 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:20.201856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01c79d-2e90-4089-a932-9b52db38fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:20.202187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01c79d-2e90-4089-a932-9b52db38fc78 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:20.202817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01c79d-2e90-4089-a932-9b52db38fc78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:20.204900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01c79d-2e90-4089-a932-9b52db38fc78 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:20.312310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 880/1761] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:20 2022] POST /identity/v3/projects => generated 409 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:20.313143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f53422-93d0-4bd0-9592-a7002f7f0747 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:20.339472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f53422-93d0-4bd0-9592-a7002f7f0747 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:20.424530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a44e48-f6b3-46e0-bb8c-d05e5517ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:20.425246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a44e48-f6b3-46e0-bb8c-d05e5517ed33 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:20.425717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a44e48-f6b3-46e0-bb8c-d05e5517ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:20.426149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a44e48-f6b3-46e0-bb8c-d05e5517ed33 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:20.426872 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a44e48-f6b3-46e0-bb8c-d05e5517ed33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:20.428885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a44e48-f6b3-46e0-bb8c-d05e5517ed33 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:20.536883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01c79d-2e90-4089-a932-9b52db38fc78 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:20.537360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 882/1762] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:20 2022] POST /identity/v3/users => generated 311 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:20.553423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01c79d-2e90-4089-a932-9b52db38fc78 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:20.563180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd95a6a-b909-4bf5-9a39-604b34faa894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:20.564115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd95a6a-b909-4bf5-9a39-604b34faa894 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:20.564539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd95a6a-b909-4bf5-9a39-604b34faa894 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:20.565251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd95a6a-b909-4bf5-9a39-604b34faa894 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:20.566094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd95a6a-b909-4bf5-9a39-604b34faa894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:20.568022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd95a6a-b909-4bf5-9a39-604b34faa894 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:20.588420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 881/1763] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:20.590042 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a44e48-f6b3-46e0-bb8c-d05e5517ed33 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:20.600570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a44e48-f6b3-46e0-bb8c-d05e5517ed33 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:20.605200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10939b-cab4-4c63-bb71-ad6115e0fff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:20.605872 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10939b-cab4-4c63-bb71-ad6115e0fff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:20.606555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10939b-cab4-4c63-bb71-ad6115e0fff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:20.606675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10939b-cab4-4c63-bb71-ad6115e0fff6 admin admin] PATH_INFO: `/v3/projects/42241183813f4de49405e2e5a85eaa40/users/e97eeb17151643d5b6f7abce3e8835c5/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:20.623562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10939b-cab4-4c63-bb71-ad6115e0fff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42241183813f4de49405e2e5a85eaa40, user_id=e97eeb17151643d5b6f7abce3e8835c5, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:20.626585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10939b-cab4-4c63-bb71-ad6115e0fff6 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:20.708604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 883/1764] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:20 2022] PUT /identity/v3/projects/42241183813f4de49405e2e5a85eaa40/users/e97eeb17151643d5b6f7abce3e8835c5/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:20.712390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd95a6a-b909-4bf5-9a39-604b34faa894 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:20.722553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd95a6a-b909-4bf5-9a39-604b34faa894 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:20.781208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23afb128-9d72-4579-9345-0bab44a0cb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:20.782090 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afb128-9d72-4579-9345-0bab44a0cb1f admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:20.782471 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afb128-9d72-4579-9345-0bab44a0cb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:20.782893 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23afb128-9d72-4579-9345-0bab44a0cb1f admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:20.784069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afb128-9d72-4579-9345-0bab44a0cb1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:20.786217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23afb128-9d72-4579-9345-0bab44a0cb1f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:20.799463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 882/1765] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:20.804218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10939b-cab4-4c63-bb71-ad6115e0fff6 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:20.815111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10939b-cab4-4c63-bb71-ad6115e0fff6 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:20.820097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3a4b2-7a54-4bc8-9bde-946729ac9736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:20.820864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3a4b2-7a54-4bc8-9bde-946729ac9736 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:20.821240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3a4b2-7a54-4bc8-9bde-946729ac9736 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:20.821574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3a4b2-7a54-4bc8-9bde-946729ac9736 admin admin] PATH_INFO: `/v3/projects/42241183813f4de49405e2e5a85eaa40/users/e97eeb17151643d5b6f7abce3e8835c5/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:20.823581 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3a4b2-7a54-4bc8-9bde-946729ac9736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42241183813f4de49405e2e5a85eaa40, user_id=e97eeb17151643d5b6f7abce3e8835c5, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:20.827446 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3a4b2-7a54-4bc8-9bde-946729ac9736 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:20.840211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 884/1766] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:20 2022] PUT /identity/v3/projects/42241183813f4de49405e2e5a85eaa40/users/e97eeb17151643d5b6f7abce3e8835c5/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:20.851477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac18e0-a6f9-476d-93f8-2ff958c1a2b3 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:20.851768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac18e0-a6f9-476d-93f8-2ff958c1a2b3 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:20.852037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fac18e0-a6f9-476d-93f8-2ff958c1a2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:20.880765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9fac18e0-a6f9-476d-93f8-2ff958c1a2b3 None None] MFA Rules not processed for user `e97eeb17151643d5b6f7abce3e8835c5`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:20.881982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9fac18e0-a6f9-476d-93f8-2ff958c1a2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:20.959901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 883/1767] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:20 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:20.983827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3a4b2-7a54-4bc8-9bde-946729ac9736 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:20.987697 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-46a3a4b2-7a54-4bc8-9bde-946729ac9736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:21.055967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3a4b2-7a54-4bc8-9bde-946729ac9736 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:21.059782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08899262-2cf1-4624-878c-9e40c7f389c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:21.061670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08899262-2cf1-4624-878c-9e40c7f389c1 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:21.061945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08899262-2cf1-4624-878c-9e40c7f389c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:21.062239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08899262-2cf1-4624-878c-9e40c7f389c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:21.068441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:21.072045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08899262-2cf1-4624-878c-9e40c7f389c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:21.073466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08899262-2cf1-4624-878c-9e40c7f389c1 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:21.084252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 885/1768] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:20 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:21.440448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac18e0-a6f9-476d-93f8-2ff958c1a2b3 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:21.451729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fac18e0-a6f9-476d-93f8-2ff958c1a2b3 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:21.456020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8b15e-433f-404e-bbf1-ba65ee5bd306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:21.463564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8b15e-433f-404e-bbf1-ba65ee5bd306 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:21.463564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8b15e-433f-404e-bbf1-ba65ee5bd306 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:21.463564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8b15e-433f-404e-bbf1-ba65ee5bd306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:21.510744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:21.523999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8b15e-433f-404e-bbf1-ba65ee5bd306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:21.525436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8b15e-433f-404e-bbf1-ba65ee5bd306 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:21.535415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 884/1769] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:21 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:22.673176 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08899262-2cf1-4624-878c-9e40c7f389c1 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:22.681570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08899262-2cf1-4624-878c-9e40c7f389c1 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:22.685450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6b341-e3e5-4647-854e-36cba6d01066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:22.686065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6b341-e3e5-4647-854e-36cba6d01066 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:22.686336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6b341-e3e5-4647-854e-36cba6d01066 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:22.686598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6b341-e3e5-4647-854e-36cba6d01066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:22.688351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f5c6b341-e3e5-4647-854e-36cba6d01066 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:22.746389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6b341-e3e5-4647-854e-36cba6d01066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:22.747842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6b341-e3e5-4647-854e-36cba6d01066 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:22.756643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 886/1770] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:22 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:52:31.633668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=9 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:32.752571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=31 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:38.202474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b7bea-d096-44e3-bcb7-95c1692b45bc None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:38.203952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b7bea-d096-44e3-bcb7-95c1692b45bc None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:38.204468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b7bea-d096-44e3-bcb7-95c1692b45bc None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:38.206987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 885/1771] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 16:52:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:52:38.234966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a59a7a-b864-4278-bd3c-accfb2a30c8d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:38.235884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a59a7a-b864-4278-bd3c-accfb2a30c8d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:38.236356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a59a7a-b864-4278-bd3c-accfb2a30c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:38.262775 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b2a59a7a-b864-4278-bd3c-accfb2a30c8d None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:38.264395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b2a59a7a-b864-4278-bd3c-accfb2a30c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:38.348696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 887/1772] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 16:52:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:38.375106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b7bea-d096-44e3-bcb7-95c1692b45bc None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:38.377884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-467b7bea-d096-44e3-bcb7-95c1692b45bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:38.404269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b7bea-d096-44e3-bcb7-95c1692b45bc None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:38.411076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d22e4-82d7-4ae4-89df-cdb9a6c57fdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:38.411076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d22e4-82d7-4ae4-89df-cdb9a6c57fdb service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:38.411076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d22e4-82d7-4ae4-89df-cdb9a6c57fdb service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:38.411076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d22e4-82d7-4ae4-89df-cdb9a6c57fdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:38.424785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d22e4-82d7-4ae4-89df-cdb9a6c57fdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:38.427673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d22e4-82d7-4ae4-89df-cdb9a6c57fdb service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:38.440415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 886/1773] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:41.097615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a59a7a-b864-4278-bd3c-accfb2a30c8d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:41.099152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b2a59a7a-b864-4278-bd3c-accfb2a30c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:41.171779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a59a7a-b864-4278-bd3c-accfb2a30c8d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:41.186556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:41.187431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:41.188128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:41.188712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] PATH_INFO: `/v3/users/f0842f75784e457499df6a0273a19120` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:41.190224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0842f75784e457499df6a0273a19120)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:41.192470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:41.299062 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d22e4-82d7-4ae4-89df-cdb9a6c57fdb service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:41.309876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d22e4-82d7-4ae4-89df-cdb9a6c57fdb service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:41.315475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:41.316107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:41.316409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:41.316686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] PATH_INFO: `/v3/users/969f8bc72e3e4295b90eb7eb0cffa50e` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:41.317864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=969f8bc72e3e4295b90eb7eb0cffa50e)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:41.319836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:41.375766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0842f75784e457499df6a0273a19120'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:41.449055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0842f75784e457499df6a0273a19120'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:41.463378 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0842f75784e457499df6a0273a19120'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:41.464078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0842f75784e457499df6a0273a19120'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:41.464744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ffd48700-0b0a-4a4f-8e1a-065949d82b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0842f75784e457499df6a0273a19120'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:41.477783 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 888/1774] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:41 2022] DELETE /identity/v3/users/f0842f75784e457499df6a0273a19120 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:41.482273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c83635-5cb9-479e-96c6-5b34c9ba777d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:41.482713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c83635-5cb9-479e-96c6-5b34c9ba777d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:41.482828 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c83635-5cb9-479e-96c6-5b34c9ba777d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:41.502908 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d7c83635-5cb9-479e-96c6-5b34c9ba777d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:41.504121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d7c83635-5cb9-479e-96c6-5b34c9ba777d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:41.510310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:41.517472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '969f8bc72e3e4295b90eb7eb0cffa50e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:41.578547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 889/1775] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:52:41.592427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c83635-5cb9-479e-96c6-5b34c9ba777d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:41.593924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d7c83635-5cb9-479e-96c6-5b34c9ba777d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:41.602738 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '969f8bc72e3e4295b90eb7eb0cffa50e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:41.610079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '969f8bc72e3e4295b90eb7eb0cffa50e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:41.622510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '969f8bc72e3e4295b90eb7eb0cffa50e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:41.623398 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca6bc839-0554-4909-a50b-b739eeef7ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '969f8bc72e3e4295b90eb7eb0cffa50e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:41.630164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 887/1776] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:41 2022] DELETE /identity/v3/users/969f8bc72e3e4295b90eb7eb0cffa50e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:52:41.634693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7bcb-4231-48de-bd59-316cb8aa5303 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:41.635102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7bcb-4231-48de-bd59-316cb8aa5303 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:41.635403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e7bcb-4231-48de-bd59-316cb8aa5303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:41.654184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c83635-5cb9-479e-96c6-5b34c9ba777d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:41.656243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-185e7bcb-4231-48de-bd59-316cb8aa5303 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:41.657447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-185e7bcb-4231-48de-bd59-316cb8aa5303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:41.715277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2af1a6-4a66-4497-bb0a-730fb37da0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:41.716229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2af1a6-4a66-4497-bb0a-730fb37da0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:41.716500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2af1a6-4a66-4497-bb0a-730fb37da0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:41.716764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2af1a6-4a66-4497-bb0a-730fb37da0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:41.718929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5b2af1a6-4a66-4497-bb0a-730fb37da0d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:41.729053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 888/1777] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:52:41.735718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2af1a6-4a66-4497-bb0a-730fb37da0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:41.737340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2af1a6-4a66-4497-bb0a-730fb37da0d6 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:41.742399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e7bcb-4231-48de-bd59-316cb8aa5303 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:41.742834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-185e7bcb-4231-48de-bd59-316cb8aa5303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:41.750029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 890/1778] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:52:41.757410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e7bcb-4231-48de-bd59-316cb8aa5303 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:41.765239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdec79-08fa-4feb-8795-6be7eb7e5325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:41.766236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdec79-08fa-4feb-8795-6be7eb7e5325 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:41.767963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdec79-08fa-4feb-8795-6be7eb7e5325 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:41.767963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdec79-08fa-4feb-8795-6be7eb7e5325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:41.781713 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdec79-08fa-4feb-8795-6be7eb7e5325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:41.784449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdec79-08fa-4feb-8795-6be7eb7e5325 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:41.795344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 889/1779] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:52:42.344245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2af1a6-4a66-4497-bb0a-730fb37da0d6 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:42.345699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5b2af1a6-4a66-4497-bb0a-730fb37da0d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:42.360671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2af1a6-4a66-4497-bb0a-730fb37da0d6 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:42.368593 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:42.368593 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:42.368593 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:42.368593 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] PATH_INFO: `/v3/projects/37616e70da8f46afa47a6e367c82d459` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:42.369411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37616e70da8f46afa47a6e367c82d459)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:42.371939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:42.429225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdec79-08fa-4feb-8795-6be7eb7e5325 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:42.430756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-adfdec79-08fa-4feb-8795-6be7eb7e5325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:42.453504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdec79-08fa-4feb-8795-6be7eb7e5325 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:42.460071 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:42.460593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:42.460849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:42.461123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] PATH_INFO: `/v3/projects/0a27a217d6874b6a8ee7eb932c2509f5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:42.462307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a27a217d6874b6a8ee7eb932c2509f5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:42.465289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:42.514968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37616e70da8f46afa47a6e367c82d459', 'request_id': 'req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:42.631620 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37616e70da8f46afa47a6e367c82d459', 'request_id': 'req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:42.641461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37616e70da8f46afa47a6e367c82d459', 'request_id': 'req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:42.646500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:42.657400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] The token cache is being invalidate because project 37616e70da8f46afa47a6e367c82d459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:52:42.657776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:42.658371 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:42.665799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 891/1780] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:42 2022] DELETE /identity/v3/projects/37616e70da8f46afa47a6e367c82d459 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:52:42.672659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:42.672659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:42.675246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:42.682324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a27a217d6874b6a8ee7eb932c2509f5', 'request_id': 'req-940796b2-962d-4643-a1e8-04d2f9be6d25'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:42.752043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cce8d-acab-46c1-8bee-d0e2b9fb7472 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:42.781129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a27a217d6874b6a8ee7eb932c2509f5', 'request_id': 'req-940796b2-962d-4643-a1e8-04d2f9be6d25'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:42.792344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a27a217d6874b6a8ee7eb932c2509f5', 'request_id': 'req-940796b2-962d-4643-a1e8-04d2f9be6d25'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:42.806184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] The token cache is being invalidate because project 0a27a217d6874b6a8ee7eb932c2509f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:52:42.806771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:42.808545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:42.814044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 890/1781] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:42 2022] DELETE /identity/v3/projects/0a27a217d6874b6a8ee7eb932c2509f5 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:52:42.821248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:42.823589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:42.833118 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:42.833884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:42.834285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:42.836010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] PATH_INFO: `/v3/users/e97eeb17151643d5b6f7abce3e8835c5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:42.836779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e97eeb17151643d5b6f7abce3e8835c5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:42.839418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:42.913843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940796b2-962d-4643-a1e8-04d2f9be6d25 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:42.919025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:42.919649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:42.919931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:42.920208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] PATH_INFO: `/v3/users/dde11155099e4733978b458841ffff7b` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:42.921411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dde11155099e4733978b458841ffff7b)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:42.923603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:43.080351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e97eeb17151643d5b6f7abce3e8835c5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:43.149251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dde11155099e4733978b458841ffff7b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:43.168751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e97eeb17151643d5b6f7abce3e8835c5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:43.182914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e97eeb17151643d5b6f7abce3e8835c5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:43.183634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e97eeb17151643d5b6f7abce3e8835c5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:43.184287 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e97eeb17151643d5b6f7abce3e8835c5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:43.199950 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 892/1782] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:42 2022] DELETE /identity/v3/users/e97eeb17151643d5b6f7abce3e8835c5 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:43.266334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dde11155099e4733978b458841ffff7b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:43.274372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dde11155099e4733978b458841ffff7b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:43.298395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dde11155099e4733978b458841ffff7b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:43.300943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dde11155099e4733978b458841ffff7b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:43.308972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 891/1783] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:42 2022] DELETE /identity/v3/users/dde11155099e4733978b458841ffff7b => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:52:43.811998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:43.813810 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:43.857826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:43.980586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcdf68-bed8-45aa-b7cc-9e2a52be064e admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:44.094124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:44.094998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:44.095383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:44.095759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] PATH_INFO: `/v3/projects/42241183813f4de49405e2e5a85eaa40` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:44.097098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42241183813f4de49405e2e5a85eaa40)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:44.099518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:44.109327 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:44.111113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:44.140119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff285c-d9ae-438c-b104-f8d0277133b3 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:44.148067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:44.149284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:44.150626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:44.151107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] PATH_INFO: `/v3/projects/d60eafc840bf413cb0e70d2ed34a7256` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:44.152645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d60eafc840bf413cb0e70d2ed34a7256)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:44.157690 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:44.258213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42241183813f4de49405e2e5a85eaa40', 'request_id': 'req-7dcc0890-4570-4107-b59b-83005cbb6b47'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:44.340900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:44.354704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd60eafc840bf413cb0e70d2ed34a7256', 'request_id': 'req-669ad09e-c4e5-479e-ad4e-4f04c27e041e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:44.363871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42241183813f4de49405e2e5a85eaa40', 'request_id': 'req-7dcc0890-4570-4107-b59b-83005cbb6b47'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:44.370925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42241183813f4de49405e2e5a85eaa40', 'request_id': 'req-7dcc0890-4570-4107-b59b-83005cbb6b47'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:44.389676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] The token cache is being invalidate because project 42241183813f4de49405e2e5a85eaa40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:52:44.393442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:44.393442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7dcc0890-4570-4107-b59b-83005cbb6b47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:44.394723 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 893/1784] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:43 2022] DELETE /identity/v3/projects/42241183813f4de49405e2e5a85eaa40 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:44.398038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f04bcd-44a7-40f2-9027-10233f445889 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:44.398298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f04bcd-44a7-40f2-9027-10233f445889 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:44.398670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f04bcd-44a7-40f2-9027-10233f445889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:44.415181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd60eafc840bf413cb0e70d2ed34a7256', 'request_id': 'req-669ad09e-c4e5-479e-ad4e-4f04c27e041e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:44.422484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-24f04bcd-44a7-40f2-9027-10233f445889 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:44.427782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-24f04bcd-44a7-40f2-9027-10233f445889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:44.428031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd60eafc840bf413cb0e70d2ed34a7256', 'request_id': 'req-669ad09e-c4e5-479e-ad4e-4f04c27e041e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:44.460432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] The token cache is being invalidate because project d60eafc840bf413cb0e70d2ed34a7256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:52:44.461239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:44.461475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:44.463347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 892/1785] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:44 2022] DELETE /identity/v3/projects/d60eafc840bf413cb0e70d2ed34a7256 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:44.470167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:44.474741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:44.537388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 894/1786] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:44.548858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f78118-08fb-46c1-98bd-8b9476f30071 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:44.549443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f78118-08fb-46c1-98bd-8b9476f30071 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:44.549756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f78118-08fb-46c1-98bd-8b9476f30071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:44.594694 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-46f78118-08fb-46c1-98bd-8b9476f30071 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:44.609523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-46f78118-08fb-46c1-98bd-8b9476f30071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:44.627923 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 895/1787] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:52:44.643895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f78118-08fb-46c1-98bd-8b9476f30071 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:44.656161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ad09e-c4e5-479e-ad4e-4f04c27e041e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:44.658568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f78118-08fb-46c1-98bd-8b9476f30071 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:44.662823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:44.663722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:44.664405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:44.664861 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] PATH_INFO: `/v3/users/4ee0365fa573429cba83d1ee2782bfff` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:44.664968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908208a5-db00-4085-ab0f-923646301da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:44.665624 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908208a5-db00-4085-ab0f-923646301da8 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:44.666007 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908208a5-db00-4085-ab0f-923646301da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:44.666374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908208a5-db00-4085-ab0f-923646301da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:44.667198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908208a5-db00-4085-ab0f-923646301da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:52:44.667564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908208a5-db00-4085-ab0f-923646301da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:44.667866 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee0365fa573429cba83d1ee2782bfff)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:44.669430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908208a5-db00-4085-ab0f-923646301da8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:44.670926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:44.691751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 896/1788] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:52:44.692841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908208a5-db00-4085-ab0f-923646301da8 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:44.723820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908208a5-db00-4085-ab0f-923646301da8 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:44.736991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a2f54-c0e8-483b-8de4-ed4d9fa500f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:44.737610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a2f54-c0e8-483b-8de4-ed4d9fa500f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:44.738034 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a2f54-c0e8-483b-8de4-ed4d9fa500f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:44.738621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a2f54-c0e8-483b-8de4-ed4d9fa500f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:44.739522 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a2f54-c0e8-483b-8de4-ed4d9fa500f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:44.741796 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a2f54-c0e8-483b-8de4-ed4d9fa500f5 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:44.898223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 897/1789] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:44 2022] POST /identity/v3/projects => generated 365 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:44.902803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a2f54-c0e8-483b-8de4-ed4d9fa500f5 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:44.977491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee0365fa573429cba83d1ee2782bfff'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:45.027237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a2f54-c0e8-483b-8de4-ed4d9fa500f5 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:45.035051 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd6972-3ab2-4a06-956c-c033cf0cd744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:45.036961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd6972-3ab2-4a06-956c-c033cf0cd744 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:45.037571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd6972-3ab2-4a06-956c-c033cf0cd744 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:45.038174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bd6972-3ab2-4a06-956c-c033cf0cd744 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:45.039094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd6972-3ab2-4a06-956c-c033cf0cd744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:45.042021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bd6972-3ab2-4a06-956c-c033cf0cd744 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:45.053903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee0365fa573429cba83d1ee2782bfff'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:45.074872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee0365fa573429cba83d1ee2782bfff'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:45.105312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee0365fa573429cba83d1ee2782bfff'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:45.107235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee0365fa573429cba83d1ee2782bfff'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:45.121782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 893/1790] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:44 2022] DELETE /identity/v3/users/4ee0365fa573429cba83d1ee2782bfff => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:45.182306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 898/1791] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:44 2022] POST /identity/v3/users => generated 283 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:52:45.187136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:45.189768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:45.307954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d0052-2493-462e-a27f-cbc05bb91c20 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:45.320298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8da062-510f-41e2-9534-12ddd72f87a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:45.321082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8da062-510f-41e2-9534-12ddd72f87a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:45.321436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8da062-510f-41e2-9534-12ddd72f87a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:45.322246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8da062-510f-41e2-9534-12ddd72f87a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:45.322984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8da062-510f-41e2-9534-12ddd72f87a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:45.325215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8da062-510f-41e2-9534-12ddd72f87a4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:45.341621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 894/1792] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:45.345921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd6972-3ab2-4a06-956c-c033cf0cd744 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:45.357769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bd6972-3ab2-4a06-956c-c033cf0cd744 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:45.362769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5ffe5-04a1-4221-8cdc-b6b4f3526cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:45.363372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ffe5-04a1-4221-8cdc-b6b4f3526cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:45.363651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ffe5-04a1-4221-8cdc-b6b4f3526cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:45.363919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5ffe5-04a1-4221-8cdc-b6b4f3526cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:45.364560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5ffe5-04a1-4221-8cdc-b6b4f3526cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:45.366177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5ffe5-04a1-4221-8cdc-b6b4f3526cc4 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:45.385439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 899/1793] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:45.386924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8da062-510f-41e2-9534-12ddd72f87a4 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:45.398528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8da062-510f-41e2-9534-12ddd72f87a4 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:45.404816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63498f74-1680-4ba8-ae01-cbe2e09168aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:45.405756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63498f74-1680-4ba8-ae01-cbe2e09168aa admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:45.406143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63498f74-1680-4ba8-ae01-cbe2e09168aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:45.406508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63498f74-1680-4ba8-ae01-cbe2e09168aa admin admin] PATH_INFO: `/v3/projects/0b02d80e089d4deabe026086ff5c09cd/users/18be4d5dfad1481aa2b38331d68acf6b/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:45.426003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63498f74-1680-4ba8-ae01-cbe2e09168aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b02d80e089d4deabe026086ff5c09cd, user_id=18be4d5dfad1481aa2b38331d68acf6b, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:45.428690 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63498f74-1680-4ba8-ae01-cbe2e09168aa admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:45.530070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 895/1794] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:45 2022] PUT /identity/v3/projects/0b02d80e089d4deabe026086ff5c09cd/users/18be4d5dfad1481aa2b38331d68acf6b/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:45.548425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765ed3-2721-47d0-a71b-6dcc5d45690c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:45.548710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765ed3-2721-47d0-a71b-6dcc5d45690c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:45.548985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea765ed3-2721-47d0-a71b-6dcc5d45690c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:45.578547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ea765ed3-2721-47d0-a71b-6dcc5d45690c None None] MFA Rules not processed for user `18be4d5dfad1481aa2b38331d68acf6b`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:45.579766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ea765ed3-2721-47d0-a71b-6dcc5d45690c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:45.669421 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 900/1795] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:45.674537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63498f74-1680-4ba8-ae01-cbe2e09168aa admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:45.714196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:45.757662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63498f74-1680-4ba8-ae01-cbe2e09168aa admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:45.782202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8982b18-4037-4813-92c9-d6b01b07be6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:45.783172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8982b18-4037-4813-92c9-d6b01b07be6d admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:45.783561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8982b18-4037-4813-92c9-d6b01b07be6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:45.784078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8982b18-4037-4813-92c9-d6b01b07be6d admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:45.785501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8982b18-4037-4813-92c9-d6b01b07be6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:45.788090 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8982b18-4037-4813-92c9-d6b01b07be6d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:45.919437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 896/1796] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:45 2022] POST /identity/v3/projects => generated 367 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:45.924357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea765ed3-2721-47d0-a71b-6dcc5d45690c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:45.935827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea765ed3-2721-47d0-a71b-6dcc5d45690c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:45.997645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd8a7-76d5-4e02-9387-d3d38f60a5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:45.998359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd8a7-76d5-4e02-9387-d3d38f60a5ee admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:45.998631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd8a7-76d5-4e02-9387-d3d38f60a5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:45.998904 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd8a7-76d5-4e02-9387-d3d38f60a5ee admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:45.999514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bd8a7-76d5-4e02-9387-d3d38f60a5ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.001579 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bd8a7-76d5-4e02-9387-d3d38f60a5ee admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:46.103316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8982b18-4037-4813-92c9-d6b01b07be6d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:46.104166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 901/1797] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:45 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:46.104919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d8982b18-4037-4813-92c9-d6b01b07be6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:46.109367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd8a7-76d5-4e02-9387-d3d38f60a5ee admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:46.120046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8982b18-4037-4813-92c9-d6b01b07be6d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:46.127020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:46.128176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.128579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.129089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] PATH_INFO: `/v3/projects/2a4d2b10cd0e4f5da750a2b42ab841f5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.130685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a4d2b10cd0e4f5da750a2b42ab841f5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.130957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd8a7-76d5-4e02-9387-d3d38f60a5ee admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:46.133751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:46.160217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec670b6-c865-4259-96da-37c209f12792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:46.161280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec670b6-c865-4259-96da-37c209f12792 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.161870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec670b6-c865-4259-96da-37c209f12792 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.162563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec670b6-c865-4259-96da-37c209f12792 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.162987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec670b6-c865-4259-96da-37c209f12792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.166667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec670b6-c865-4259-96da-37c209f12792 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:46.192530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 902/1798] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:46.196800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec670b6-c865-4259-96da-37c209f12792 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:46.209558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec670b6-c865-4259-96da-37c209f12792 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:46.214209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07a759-db26-4427-900b-44296149a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:46.214865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07a759-db26-4427-900b-44296149a769 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.215137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07a759-db26-4427-900b-44296149a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.215408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07a759-db26-4427-900b-44296149a769 admin admin] PATH_INFO: `/v3/projects/b71a7dc047854ec0b0b9a387dd1d9ace/users/148702c56f1d4b3b92b6afe4989d9220/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.237992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07a759-db26-4427-900b-44296149a769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71a7dc047854ec0b0b9a387dd1d9ace, user_id=148702c56f1d4b3b92b6afe4989d9220, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.241536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07a759-db26-4427-900b-44296149a769 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:46.247068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a4d2b10cd0e4f5da750a2b42ab841f5', 'request_id': 'req-0e2a58b7-766d-4eea-9a86-f9ffba13293e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:46.311817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a4d2b10cd0e4f5da750a2b42ab841f5', 'request_id': 'req-0e2a58b7-766d-4eea-9a86-f9ffba13293e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:46.312789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 903/1799] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:46 2022] PUT /identity/v3/projects/b71a7dc047854ec0b0b9a387dd1d9ace/users/148702c56f1d4b3b92b6afe4989d9220/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:46.317652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07a759-db26-4427-900b-44296149a769 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:46.326127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a4d2b10cd0e4f5da750a2b42ab841f5', 'request_id': 'req-0e2a58b7-766d-4eea-9a86-f9ffba13293e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:46.336179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:46.341823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] The token cache is being invalidate because project 2a4d2b10cd0e4f5da750a2b42ab841f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:52:46.343717 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:46.344586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0e2a58b7-766d-4eea-9a86-f9ffba13293e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:46.346472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 897/1800] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:46 2022] DELETE /identity/v3/projects/2a4d2b10cd0e4f5da750a2b42ab841f5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:46.352601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d1b74-5f3d-4a03-93eb-a325f6d13202 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.353563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d1b74-5f3d-4a03-93eb-a325f6d13202 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.353940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d1b74-5f3d-4a03-93eb-a325f6d13202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.390561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-645d1b74-5f3d-4a03-93eb-a325f6d13202 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:46.391791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-645d1b74-5f3d-4a03-93eb-a325f6d13202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:46.426107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07a759-db26-4427-900b-44296149a769 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:46.432709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27a173-21dc-4231-9e8a-cd6ae2b8dd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:46.434408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27a173-21dc-4231-9e8a-cd6ae2b8dd2b admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.434726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27a173-21dc-4231-9e8a-cd6ae2b8dd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.435039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27a173-21dc-4231-9e8a-cd6ae2b8dd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.436124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27a173-21dc-4231-9e8a-cd6ae2b8dd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.437667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27a173-21dc-4231-9e8a-cd6ae2b8dd2b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:46.466397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 904/1801] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:46.471524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27a173-21dc-4231-9e8a-cd6ae2b8dd2b admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:46.472979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2d27a173-21dc-4231-9e8a-cd6ae2b8dd2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:46.487354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27a173-21dc-4231-9e8a-cd6ae2b8dd2b admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:46.492517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e41eba-11ec-4c7c-b783-f9ea3b744c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:46.493226 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e41eba-11ec-4c7c-b783-f9ea3b744c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.493555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e41eba-11ec-4c7c-b783-f9ea3b744c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.493772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e41eba-11ec-4c7c-b783-f9ea3b744c2f admin admin] PATH_INFO: `/v3/projects/b71a7dc047854ec0b0b9a387dd1d9ace/users/148702c56f1d4b3b92b6afe4989d9220/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.496109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e41eba-11ec-4c7c-b783-f9ea3b744c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71a7dc047854ec0b0b9a387dd1d9ace, user_id=148702c56f1d4b3b92b6afe4989d9220, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.499074 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e41eba-11ec-4c7c-b783-f9ea3b744c2f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:46.500571 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 898/1802] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:52:46.508155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bb675-35e4-4526-8ef2-31bf398a3a40 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.508606 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bb675-35e4-4526-8ef2-31bf398a3a40 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.508878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508bb675-35e4-4526-8ef2-31bf398a3a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.510242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 905/1803] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:46 2022] PUT /identity/v3/projects/b71a7dc047854ec0b0b9a387dd1d9ace/users/148702c56f1d4b3b92b6afe4989d9220/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:52:46.521717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca9ce2-baec-484f-9d8b-f6b9f2a0a63c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.522258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca9ce2-baec-484f-9d8b-f6b9f2a0a63c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.522530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca9ce2-baec-484f-9d8b-f6b9f2a0a63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.531220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-508bb675-35e4-4526-8ef2-31bf398a3a40 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:46.531965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-508bb675-35e4-4526-8ef2-31bf398a3a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:46.557994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-03ca9ce2-baec-484f-9d8b-f6b9f2a0a63c None None] MFA Rules not processed for user `148702c56f1d4b3b92b6afe4989d9220`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:46.559786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-03ca9ce2-baec-484f-9d8b-f6b9f2a0a63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:46.606149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 899/1804] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:52:46.609647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bb675-35e4-4526-8ef2-31bf398a3a40 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:46.619426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508bb675-35e4-4526-8ef2-31bf398a3a40 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:46.625101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63979b-233d-4294-8aff-86077561eda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:46.625701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63979b-233d-4294-8aff-86077561eda6 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.625996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63979b-233d-4294-8aff-86077561eda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.626270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63979b-233d-4294-8aff-86077561eda6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.628817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63979b-233d-4294-8aff-86077561eda6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:52:46.628817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63979b-233d-4294-8aff-86077561eda6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.629359 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63979b-233d-4294-8aff-86077561eda6 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:46.645555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 900/1805] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:52:46.646746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63979b-233d-4294-8aff-86077561eda6 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:46.651963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 906/1806] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:52:46.660743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63979b-233d-4294-8aff-86077561eda6 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:46.664253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca9ce2-baec-484f-9d8b-f6b9f2a0a63c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:46.666325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-03ca9ce2-baec-484f-9d8b-f6b9f2a0a63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:46.666684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eca64d-c271-4c87-a2c6-5101b58182f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:46.667400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eca64d-c271-4c87-a2c6-5101b58182f1 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.667795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eca64d-c271-4c87-a2c6-5101b58182f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.668164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eca64d-c271-4c87-a2c6-5101b58182f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.668889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eca64d-c271-4c87-a2c6-5101b58182f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.672979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eca64d-c271-4c87-a2c6-5101b58182f1 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:46.734646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca9ce2-baec-484f-9d8b-f6b9f2a0a63c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:46.738583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1fe8a-2dc6-46de-b8c2-359c6905ceb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:46.739118 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1fe8a-2dc6-46de-b8c2-359c6905ceb5 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.739390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1fe8a-2dc6-46de-b8c2-359c6905ceb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.739655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1fe8a-2dc6-46de-b8c2-359c6905ceb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.749309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1fe8a-2dc6-46de-b8c2-359c6905ceb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.750989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1fe8a-2dc6-46de-b8c2-359c6905ceb5 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:46.761989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 901/1807] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:46 2022] POST /identity/v3/projects => generated 401 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:46.766629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eca64d-c271-4c87-a2c6-5101b58182f1 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:46.775545 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 907/1808] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:52:46.839260 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eca64d-c271-4c87-a2c6-5101b58182f1 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:46.853851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3377b8-c0c5-496b-8db9-01a6913e1022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:46.854536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3377b8-c0c5-496b-8db9-01a6913e1022 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.854865 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3377b8-c0c5-496b-8db9-01a6913e1022 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.855144 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3377b8-c0c5-496b-8db9-01a6913e1022 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.855785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3377b8-c0c5-496b-8db9-01a6913e1022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.858193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3377b8-c0c5-496b-8db9-01a6913e1022 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:46.971591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 902/1809] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:46 2022] POST /identity/v3/users => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:46.972863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1fe8a-2dc6-46de-b8c2-359c6905ceb5 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:46.983878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1fe8a-2dc6-46de-b8c2-359c6905ceb5 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:46.989038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2e26c-a855-494a-ad88-edb8c6059f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:46.989834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2e26c-a855-494a-ad88-edb8c6059f2e admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:46.994602 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2e26c-a855-494a-ad88-edb8c6059f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:46.994602 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2e26c-a855-494a-ad88-edb8c6059f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:46.994602 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2e26c-a855-494a-ad88-edb8c6059f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:46.994602 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2e26c-a855-494a-ad88-edb8c6059f2e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:47.009456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 908/1810] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:47.010640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3377b8-c0c5-496b-8db9-01a6913e1022 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:47.014599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2e26c-a855-494a-ad88-edb8c6059f2e admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:47.022045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3377b8-c0c5-496b-8db9-01a6913e1022 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:47.027316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b997c09-d631-4eee-9b75-88595b89a1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:47.028119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b997c09-d631-4eee-9b75-88595b89a1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:47.028474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b997c09-d631-4eee-9b75-88595b89a1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:47.028822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b997c09-d631-4eee-9b75-88595b89a1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:47.029596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b997c09-d631-4eee-9b75-88595b89a1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:47.035693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b997c09-d631-4eee-9b75-88595b89a1d4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:47.048734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 903/1811] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:47.059124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b997c09-d631-4eee-9b75-88595b89a1d4 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:47.070462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b997c09-d631-4eee-9b75-88595b89a1d4 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:47.075883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3779a-4e16-43fc-8e76-937f939c6e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:47.080658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3779a-4e16-43fc-8e76-937f939c6e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:47.080658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3779a-4e16-43fc-8e76-937f939c6e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:47.080658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3779a-4e16-43fc-8e76-937f939c6e4b admin admin] PATH_INFO: `/v3/projects/6854bb93d4c147f08bda5e075d86fb8b/users/4b02d198b94d4ec486727fa104d6b837/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:47.080959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2e26c-a855-494a-ad88-edb8c6059f2e admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:47.084582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189208c-bb4b-4399-b8d2-9d9d748ae2be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:47.085260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189208c-bb4b-4399-b8d2-9d9d748ae2be service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:47.085608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189208c-bb4b-4399-b8d2-9d9d748ae2be service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:47.085945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189208c-bb4b-4399-b8d2-9d9d748ae2be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:47.088001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2189208c-bb4b-4399-b8d2-9d9d748ae2be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:47.094356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3779a-4e16-43fc-8e76-937f939c6e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6854bb93d4c147f08bda5e075d86fb8b, user_id=4b02d198b94d4ec486727fa104d6b837, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:47.097093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3779a-4e16-43fc-8e76-937f939c6e4b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:47.140046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:47.157738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189208c-bb4b-4399-b8d2-9d9d748ae2be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:47.159238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189208c-bb4b-4399-b8d2-9d9d748ae2be service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:47.168214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 909/1812] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:52:47.179281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 904/1813] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:47 2022] PUT /identity/v3/projects/6854bb93d4c147f08bda5e075d86fb8b/users/4b02d198b94d4ec486727fa104d6b837/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:47.191045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f87348-9e83-4919-9a46-af7262c47775 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:47.191325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f87348-9e83-4919-9a46-af7262c47775 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:47.191590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f87348-9e83-4919-9a46-af7262c47775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:47.220073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f1f87348-9e83-4919-9a46-af7262c47775 None None] MFA Rules not processed for user `4b02d198b94d4ec486727fa104d6b837`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:47.221292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f1f87348-9e83-4919-9a46-af7262c47775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:47.294362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 910/1814] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:47 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:47.302774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3779a-4e16-43fc-8e76-937f939c6e4b admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:47.418133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3779a-4e16-43fc-8e76-937f939c6e4b admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:47.418133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb9805-b009-4d72-95c6-562c5a307502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:47.418133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb9805-b009-4d72-95c6-562c5a307502 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:47.418133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb9805-b009-4d72-95c6-562c5a307502 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:47.418631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb9805-b009-4d72-95c6-562c5a307502 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:47.418631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb9805-b009-4d72-95c6-562c5a307502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:47.418631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb9805-b009-4d72-95c6-562c5a307502 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:47.460591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:47.610335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 905/1815] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:52:47 2022] POST /identity/v3/projects => generated 399 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:52:47.614361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f87348-9e83-4919-9a46-af7262c47775 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:47.625271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f87348-9e83-4919-9a46-af7262c47775 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:47.684578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666c5da-ec18-4308-9aaf-84422267b859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:47.685193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c5da-ec18-4308-9aaf-84422267b859 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:47.685552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c5da-ec18-4308-9aaf-84422267b859 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:47.685823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c5da-ec18-4308-9aaf-84422267b859 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:47.686500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666c5da-ec18-4308-9aaf-84422267b859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:47.688462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666c5da-ec18-4308-9aaf-84422267b859 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:47.711421 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:47.781008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 911/1816] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:52:47 2022] POST /identity/v3/users => generated 306 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:52:47.785107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb9805-b009-4d72-95c6-562c5a307502 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:47.795416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb9805-b009-4d72-95c6-562c5a307502 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:47.800019 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24a2fa-19da-4a11-b99d-8c8ee9af570a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:47.800625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24a2fa-19da-4a11-b99d-8c8ee9af570a admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:47.800892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24a2fa-19da-4a11-b99d-8c8ee9af570a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:47.805505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24a2fa-19da-4a11-b99d-8c8ee9af570a admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:47.805505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24a2fa-19da-4a11-b99d-8c8ee9af570a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:47.805505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24a2fa-19da-4a11-b99d-8c8ee9af570a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:47.813701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 906/1817] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:47.817764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666c5da-ec18-4308-9aaf-84422267b859 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:47.828394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666c5da-ec18-4308-9aaf-84422267b859 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:47.832981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537405a8-fdd6-4b96-ba95-ca61d17f4c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:47.834213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537405a8-fdd6-4b96-ba95-ca61d17f4c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:47.834213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537405a8-fdd6-4b96-ba95-ca61d17f4c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:47.834213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537405a8-fdd6-4b96-ba95-ca61d17f4c1a admin admin] PATH_INFO: `/v3/projects/f4aab875fbdc4e838891cd8958638160/users/a39a2fe6b0ba4312b6497a44dd2f8f67/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:47.849730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537405a8-fdd6-4b96-ba95-ca61d17f4c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4aab875fbdc4e838891cd8958638160, user_id=a39a2fe6b0ba4312b6497a44dd2f8f67, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:47.852466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537405a8-fdd6-4b96-ba95-ca61d17f4c1a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:47.977383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 912/1818] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:47 2022] PUT /identity/v3/projects/f4aab875fbdc4e838891cd8958638160/users/a39a2fe6b0ba4312b6497a44dd2f8f67/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:47.982271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24a2fa-19da-4a11-b99d-8c8ee9af570a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:47.992901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24a2fa-19da-4a11-b99d-8c8ee9af570a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:48.053769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e189e59-be18-47af-8e1b-c9d29d935381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:48.054429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e189e59-be18-47af-8e1b-c9d29d935381 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:48.054706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e189e59-be18-47af-8e1b-c9d29d935381 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:48.054978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e189e59-be18-47af-8e1b-c9d29d935381 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:48.055680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e189e59-be18-47af-8e1b-c9d29d935381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:48.057193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e189e59-be18-47af-8e1b-c9d29d935381 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:48.073744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 907/1819] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:52:48.078094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537405a8-fdd6-4b96-ba95-ca61d17f4c1a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:48.093325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537405a8-fdd6-4b96-ba95-ca61d17f4c1a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:48.113736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:48.126878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5247dc1-4fe7-4697-9816-dc298ffbd6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:48.129336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5247dc1-4fe7-4697-9816-dc298ffbd6ed admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:48.129336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5247dc1-4fe7-4697-9816-dc298ffbd6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:48.129336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5247dc1-4fe7-4697-9816-dc298ffbd6ed admin admin] PATH_INFO: `/v3/projects/f4aab875fbdc4e838891cd8958638160/users/a39a2fe6b0ba4312b6497a44dd2f8f67/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:48.135105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5247dc1-4fe7-4697-9816-dc298ffbd6ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4aab875fbdc4e838891cd8958638160, user_id=a39a2fe6b0ba4312b6497a44dd2f8f67, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:48.137910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5247dc1-4fe7-4697-9816-dc298ffbd6ed admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:48.157587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 913/1820] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:52:48 2022] PUT /identity/v3/projects/f4aab875fbdc4e838891cd8958638160/users/a39a2fe6b0ba4312b6497a44dd2f8f67/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:48.169756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e9d4-a4b2-49e4-a3ec-10a78eae17b8 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:48.170215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e9d4-a4b2-49e4-a3ec-10a78eae17b8 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:48.170562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e9d4-a4b2-49e4-a3ec-10a78eae17b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:48.204437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-4672e9d4-a4b2-49e4-a3ec-10a78eae17b8 None None] MFA Rules not processed for user `a39a2fe6b0ba4312b6497a44dd2f8f67`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:48.205677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4672e9d4-a4b2-49e4-a3ec-10a78eae17b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:48.269878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:48.283781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 908/1821] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:48 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:48.295900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5247dc1-4fe7-4697-9816-dc298ffbd6ed admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:48.352014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5247dc1-4fe7-4697-9816-dc298ffbd6ed admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:48.355859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbafa55-fdcd-4c8e-8991-4dae58319e86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:48.356245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbafa55-fdcd-4c8e-8991-4dae58319e86 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:48.356642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbafa55-fdcd-4c8e-8991-4dae58319e86 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:48.356923 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbafa55-fdcd-4c8e-8991-4dae58319e86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:48.366550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbafa55-fdcd-4c8e-8991-4dae58319e86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:48.368059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbafa55-fdcd-4c8e-8991-4dae58319e86 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:48.378251 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 914/1822] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:48 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:48.402730 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:48.618117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672e9d4-a4b2-49e4-a3ec-10a78eae17b8 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:48.619753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4672e9d4-a4b2-49e4-a3ec-10a78eae17b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:48.635084 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672e9d4-a4b2-49e4-a3ec-10a78eae17b8 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:48.639500 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a837c75-a359-4565-9ab8-ac642ae1305a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:48.640105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a837c75-a359-4565-9ab8-ac642ae1305a service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:48.640385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a837c75-a359-4565-9ab8-ac642ae1305a service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:48.640653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a837c75-a359-4565-9ab8-ac642ae1305a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:48.702887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a837c75-a359-4565-9ab8-ac642ae1305a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:48.704508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a837c75-a359-4565-9ab8-ac642ae1305a service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:48.715272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 909/1823] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:48 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:58.373297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:58.651263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbafa55-fdcd-4c8e-8991-4dae58319e86 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:58.709682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=34 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:58.715551 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbafa55-fdcd-4c8e-8991-4dae58319e86 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:58.720670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:58.721293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:58.721573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:58.721845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] PATH_INFO: `/v3/users/18be4d5dfad1481aa2b38331d68acf6b` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:58.723006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] RBAC: Authorizing `identity:delete_user(user_id=18be4d5dfad1481aa2b38331d68acf6b)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:58.724906 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:58.964160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18be4d5dfad1481aa2b38331d68acf6b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:59.036402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18be4d5dfad1481aa2b38331d68acf6b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:59.051441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18be4d5dfad1481aa2b38331d68acf6b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:59.052179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18be4d5dfad1481aa2b38331d68acf6b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:59.052813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18be4d5dfad1481aa2b38331d68acf6b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:59.069125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 915/1824] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:58 2022] DELETE /identity/v3/users/18be4d5dfad1481aa2b38331d68acf6b => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:59.071528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d5ad4-28ea-4a4a-baed-aa496536ae39 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:59.071817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d5ad4-28ea-4a4a-baed-aa496536ae39 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:59.072089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402d5ad4-28ea-4a4a-baed-aa496536ae39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:59.139848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-402d5ad4-28ea-4a4a-baed-aa496536ae39 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:59.143514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-402d5ad4-28ea-4a4a-baed-aa496536ae39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:59.241290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 910/1825] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:52:59.253353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:59.255069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:59.358148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117a4ce-0162-4aa9-afca-4c3e75eec13d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:59.358148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105b62a-b7ea-43e8-9fea-153fc0db25af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:59.358148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105b62a-b7ea-43e8-9fea-153fc0db25af service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:59.358148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105b62a-b7ea-43e8-9fea-153fc0db25af service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:59.358439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105b62a-b7ea-43e8-9fea-153fc0db25af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:59.368192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105b62a-b7ea-43e8-9fea-153fc0db25af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:59.369612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105b62a-b7ea-43e8-9fea-153fc0db25af service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:59.379753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 916/1826] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:52:59.573459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d5ad4-28ea-4a4a-baed-aa496536ae39 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:59.576051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-402d5ad4-28ea-4a4a-baed-aa496536ae39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:59.589892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402d5ad4-28ea-4a4a-baed-aa496536ae39 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:59.602077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:59.606580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:59.606580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:59.606580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] PATH_INFO: `/v3/users/4b02d198b94d4ec486727fa104d6b837` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:59.606580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b02d198b94d4ec486727fa104d6b837)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:59.606884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:59.801605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b02d198b94d4ec486727fa104d6b837'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:59.870246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105b62a-b7ea-43e8-9fea-153fc0db25af service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:59.871176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b02d198b94d4ec486727fa104d6b837'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:59.871839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2105b62a-b7ea-43e8-9fea-153fc0db25af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:59.879972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b02d198b94d4ec486727fa104d6b837'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:59.885834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105b62a-b7ea-43e8-9fea-153fc0db25af service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:52:59.891128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:52:59.892004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:59.892459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:59.892813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] PATH_INFO: `/v3/users/e0d71ca66aa84022a1481e678b7a9484` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:59.895066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d71ca66aa84022a1481e678b7a9484)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:52:59.896461 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b02d198b94d4ec486727fa104d6b837'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:59.897251 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:52:59.897766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-c47994f5-7d83-43ce-afe2-a39ce542e1f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b02d198b94d4ec486727fa104d6b837'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:52:59.911482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 911/1827] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:59 2022] DELETE /identity/v3/users/4b02d198b94d4ec486727fa104d6b837 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:52:59.915684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69357f-af39-413f-b105-cd25bac3faad None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:52:59.915972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69357f-af39-413f-b105-cd25bac3faad None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:52:59.916247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69357f-af39-413f-b105-cd25bac3faad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:52:59.916681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:52:59.936521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8f69357f-af39-413f-b105-cd25bac3faad None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:52:59.937746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8f69357f-af39-413f-b105-cd25bac3faad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:52:59.996983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 912/1828] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:52:59.998424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69357f-af39-413f-b105-cd25bac3faad None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:52:59.999695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8f69357f-af39-413f-b105-cd25bac3faad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:00.011431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69357f-af39-413f-b105-cd25bac3faad None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:00.016198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:00.016813 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:00.017134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:00.017410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] PATH_INFO: `/v3/projects/0b02d80e089d4deabe026086ff5c09cd` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:00.018543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b02d80e089d4deabe026086ff5c09cd)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:00.020605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:00.029577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:00.042777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d71ca66aa84022a1481e678b7a9484'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.100580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d71ca66aa84022a1481e678b7a9484'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.118150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d71ca66aa84022a1481e678b7a9484'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.118926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d71ca66aa84022a1481e678b7a9484'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.119615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d71ca66aa84022a1481e678b7a9484'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.131773 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b02d80e089d4deabe026086ff5c09cd', 'request_id': 'req-3a363e12-c8aa-4434-b358-d3d2642fe85a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.132545 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 917/1829] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:52:59 2022] DELETE /identity/v3/users/e0d71ca66aa84022a1481e678b7a9484 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:00.133389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:00.142413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:00.197676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe01bd7-cd5f-4204-b3eb-ea58cd5dcd27 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:00.202217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45233860-c51b-48d2-ae5a-c28c938a4a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:00.202837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45233860-c51b-48d2-ae5a-c28c938a4a5e service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:00.203181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45233860-c51b-48d2-ae5a-c28c938a4a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:00.203453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45233860-c51b-48d2-ae5a-c28c938a4a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:00.205442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-45233860-c51b-48d2-ae5a-c28c938a4a5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:00.230854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b02d80e089d4deabe026086ff5c09cd', 'request_id': 'req-3a363e12-c8aa-4434-b358-d3d2642fe85a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.241575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b02d80e089d4deabe026086ff5c09cd', 'request_id': 'req-3a363e12-c8aa-4434-b358-d3d2642fe85a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.255344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] The token cache is being invalidate because project 0b02d80e089d4deabe026086ff5c09cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:00.255576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.256486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3a363e12-c8aa-4434-b358-d3d2642fe85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.258189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 913/1830] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:52:59 2022] DELETE /identity/v3/projects/0b02d80e089d4deabe026086ff5c09cd => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:00.263128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf50f2e-7cdd-4080-93fc-a8142fc82a19 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:00.263436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf50f2e-7cdd-4080-93fc-a8142fc82a19 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:00.263698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf50f2e-7cdd-4080-93fc-a8142fc82a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:00.285710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45233860-c51b-48d2-ae5a-c28c938a4a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:00.287347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45233860-c51b-48d2-ae5a-c28c938a4a5e service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:00.303617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 918/1831] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:00.303617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45233860-c51b-48d2-ae5a-c28c938a4a5e service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:00.303617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-45233860-c51b-48d2-ae5a-c28c938a4a5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:00.315822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45233860-c51b-48d2-ae5a-c28c938a4a5e service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:00.322660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:00.323520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:00.323913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:00.324286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] PATH_INFO: `/v3/users/148702c56f1d4b3b92b6afe4989d9220` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:00.325417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=148702c56f1d4b3b92b6afe4989d9220)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:00.327685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:00.330600 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-0bf50f2e-7cdd-4080-93fc-a8142fc82a19 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:00.332406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0bf50f2e-7cdd-4080-93fc-a8142fc82a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:00.358221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 914/1832] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:00.382496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf50f2e-7cdd-4080-93fc-a8142fc82a19 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:00.386016 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0bf50f2e-7cdd-4080-93fc-a8142fc82a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:00.400167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf50f2e-7cdd-4080-93fc-a8142fc82a19 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:00.405117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84c4bc-2aad-45eb-8112-df6aea6767a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:00.406300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84c4bc-2aad-45eb-8112-df6aea6767a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:00.406627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84c4bc-2aad-45eb-8112-df6aea6767a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:00.408778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84c4bc-2aad-45eb-8112-df6aea6767a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:00.426760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84c4bc-2aad-45eb-8112-df6aea6767a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:00.428260 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84c4bc-2aad-45eb-8112-df6aea6767a0 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:00.442320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 915/1833] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:00.524297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:00.534143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148702c56f1d4b3b92b6afe4989d9220'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.642345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148702c56f1d4b3b92b6afe4989d9220'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.659868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148702c56f1d4b3b92b6afe4989d9220'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.660866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148702c56f1d4b3b92b6afe4989d9220'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.661496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148702c56f1d4b3b92b6afe4989d9220'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:00.676947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 919/1834] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:00 2022] DELETE /identity/v3/users/148702c56f1d4b3b92b6afe4989d9220 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:01.337508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84c4bc-2aad-45eb-8112-df6aea6767a0 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:01.338983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1c84c4bc-2aad-45eb-8112-df6aea6767a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:01.352180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84c4bc-2aad-45eb-8112-df6aea6767a0 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:01.389138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:01.390997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:01.431650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:01.432273 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:01.432609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:01.432859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] PATH_INFO: `/v3/projects/6854bb93d4c147f08bda5e075d86fb8b` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:01.438787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6854bb93d4c147f08bda5e075d86fb8b)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:01.448332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:01.497808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda157b3-59f3-4017-a5cc-287d5b806170 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:01.505558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:01.506423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:01.506804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:01.507216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] PATH_INFO: `/v3/projects/cff6c5637d4b49a6a40719df731e3193` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:01.508674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff6c5637d4b49a6a40719df731e3193)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:01.510977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:01.592304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6854bb93d4c147f08bda5e075d86fb8b', 'request_id': 'req-ca8b24ab-25bd-4232-97f5-ac49e7880372'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:01.667902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff6c5637d4b49a6a40719df731e3193', 'request_id': 'req-13dba410-5899-4c81-92e4-7bb58387e7fd'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:01.744910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6854bb93d4c147f08bda5e075d86fb8b', 'request_id': 'req-ca8b24ab-25bd-4232-97f5-ac49e7880372'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:01.755445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6854bb93d4c147f08bda5e075d86fb8b', 'request_id': 'req-ca8b24ab-25bd-4232-97f5-ac49e7880372'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:01.767869 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] The token cache is being invalidate because project 6854bb93d4c147f08bda5e075d86fb8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:01.768560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:01.770125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:01.775041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 916/1835] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:01 2022] DELETE /identity/v3/projects/6854bb93d4c147f08bda5e075d86fb8b => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:01.780118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:01.795357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:01.801883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff6c5637d4b49a6a40719df731e3193', 'request_id': 'req-13dba410-5899-4c81-92e4-7bb58387e7fd'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:01.809293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff6c5637d4b49a6a40719df731e3193', 'request_id': 'req-13dba410-5899-4c81-92e4-7bb58387e7fd'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:01.820366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:01.841759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] The token cache is being invalidate because project cff6c5637d4b49a6a40719df731e3193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:01.842086 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:01.843163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:01.846109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 920/1836] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:01 2022] DELETE /identity/v3/projects/cff6c5637d4b49a6a40719df731e3193 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:01.846109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:01.847879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:01.934206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b24ab-25bd-4232-97f5-ac49e7880372 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:01.944165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:01.949750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dba410-5899-4c81-92e4-7bb58387e7fd admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:01.955316 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:01.955948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:01.956225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:01.956493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] PATH_INFO: `/v3/users/a39a2fe6b0ba4312b6497a44dd2f8f67` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:01.957641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39a2fe6b0ba4312b6497a44dd2f8f67)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:01.959607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:01.995271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:01.995945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:01.996226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:01.996519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] PATH_INFO: `/v3/projects/b71a7dc047854ec0b0b9a387dd1d9ace` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:01.997685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b71a7dc047854ec0b0b9a387dd1d9ace)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:01.999695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:02.192919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b71a7dc047854ec0b0b9a387dd1d9ace', 'request_id': 'req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.193567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39a2fe6b0ba4312b6497a44dd2f8f67'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.270329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b71a7dc047854ec0b0b9a387dd1d9ace', 'request_id': 'req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.277766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b71a7dc047854ec0b0b9a387dd1d9ace', 'request_id': 'req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.289465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] The token cache is being invalidate because project b71a7dc047854ec0b0b9a387dd1d9ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:02.289830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.290564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.292219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 917/1837] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:01 2022] DELETE /identity/v3/projects/b71a7dc047854ec0b0b9a387dd1d9ace => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:02.293040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:02.294469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:02.365661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39172a-c76b-4f2a-b2c3-99f6a1425f61 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:02.370962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:02.371670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:02.374361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:02.374361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] PATH_INFO: `/v3/users/9acb41eec4d546408dae9f360ba98942` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:02.374361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9acb41eec4d546408dae9f360ba98942)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:02.376029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:02.419616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39a2fe6b0ba4312b6497a44dd2f8f67'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.432631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39a2fe6b0ba4312b6497a44dd2f8f67'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.433326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39a2fe6b0ba4312b6497a44dd2f8f67'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.433930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8e3db630-52f5-47dc-9f54-0c18554b83a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39a2fe6b0ba4312b6497a44dd2f8f67'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.457459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 921/1838] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:01 2022] DELETE /identity/v3/users/a39a2fe6b0ba4312b6497a44dd2f8f67 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:02.459144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba0279-cbf0-4230-973b-70d63833d65c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:02.459667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba0279-cbf0-4230-973b-70d63833d65c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:02.460137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba0279-cbf0-4230-973b-70d63833d65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:02.485739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-7aba0279-cbf0-4230-973b-70d63833d65c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:02.485739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-7aba0279-cbf0-4230-973b-70d63833d65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:02.607474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 922/1839] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:02.615318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab76970-b705-44e9-b08c-e8c27a92c662 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:02.616048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab76970-b705-44e9-b08c-e8c27a92c662 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:02.616486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab76970-b705-44e9-b08c-e8c27a92c662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:02.630991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9acb41eec4d546408dae9f360ba98942'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.638346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0ab76970-b705-44e9-b08c-e8c27a92c662 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:02.639630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0ab76970-b705-44e9-b08c-e8c27a92c662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:02.648046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 923/1840] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:02.652253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab76970-b705-44e9-b08c-e8c27a92c662 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:02.662337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab76970-b705-44e9-b08c-e8c27a92c662 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:02.667357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68330e46-064f-4d2b-a759-4e2eaaa98f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:02.667910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68330e46-064f-4d2b-a759-4e2eaaa98f7a admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:02.668197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68330e46-064f-4d2b-a759-4e2eaaa98f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:02.669346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68330e46-064f-4d2b-a759-4e2eaaa98f7a admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:02.669659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68330e46-064f-4d2b-a759-4e2eaaa98f7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:02.669893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68330e46-064f-4d2b-a759-4e2eaaa98f7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:02.671558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68330e46-064f-4d2b-a759-4e2eaaa98f7a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:02.683995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 924/1841] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:53:02.689720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68330e46-064f-4d2b-a759-4e2eaaa98f7a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:02.700371 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68330e46-064f-4d2b-a759-4e2eaaa98f7a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:02.705167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ffd54-d822-489b-90b5-95b08620864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:02.705799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ffd54-d822-489b-90b5-95b08620864a admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:02.706115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ffd54-d822-489b-90b5-95b08620864a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:02.706386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ffd54-d822-489b-90b5-95b08620864a admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:02.707026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ffd54-d822-489b-90b5-95b08620864a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:02.709236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ffd54-d822-489b-90b5-95b08620864a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:02.762302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9acb41eec4d546408dae9f360ba98942'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.770200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9acb41eec4d546408dae9f360ba98942'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.783481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9acb41eec4d546408dae9f360ba98942'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.784247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9acb41eec4d546408dae9f360ba98942'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:02.792726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 918/1842] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:02 2022] DELETE /identity/v3/users/9acb41eec4d546408dae9f360ba98942 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:02.836253 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:02.837184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 925/1843] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:02 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:02.837764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:02.919369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c17b1-60d9-461a-8a46-ee6b95ee300c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:02.976671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbed1b-d8bc-42aa-a9d9-7fadb398d79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:02.977340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbed1b-d8bc-42aa-a9d9-7fadb398d79d admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:02.977584 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbed1b-d8bc-42aa-a9d9-7fadb398d79d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:02.977854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbed1b-d8bc-42aa-a9d9-7fadb398d79d admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:02.978628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbed1b-d8bc-42aa-a9d9-7fadb398d79d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:02.980537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbed1b-d8bc-42aa-a9d9-7fadb398d79d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.011898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ffd54-d822-489b-90b5-95b08620864a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.013735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6c2ffd54-d822-489b-90b5-95b08620864a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:03.027432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ffd54-d822-489b-90b5-95b08620864a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.033926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.034857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.035390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.035661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] PATH_INFO: `/v3/projects/f4aab875fbdc4e838891cd8958638160` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.036875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4aab875fbdc4e838891cd8958638160)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.038897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.060800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 919/1844] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:02 2022] POST /identity/v3/users => generated 292 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:03.066003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbed1b-d8bc-42aa-a9d9-7fadb398d79d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.078960 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbed1b-d8bc-42aa-a9d9-7fadb398d79d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.083921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96ff65-3a98-44b0-b43a-a10c8f0f0d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.084569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96ff65-3a98-44b0-b43a-a10c8f0f0d5a admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.087318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96ff65-3a98-44b0-b43a-a10c8f0f0d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.087507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:03.088128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96ff65-3a98-44b0-b43a-a10c8f0f0d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.088891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96ff65-3a98-44b0-b43a-a10c8f0f0d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.091377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96ff65-3a98-44b0-b43a-a10c8f0f0d5a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.102985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 920/1845] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:03.107509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96ff65-3a98-44b0-b43a-a10c8f0f0d5a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.117291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96ff65-3a98-44b0-b43a-a10c8f0f0d5a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.122360 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6140fd4-111e-44f8-84e8-95fb3c2d2944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.123009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6140fd4-111e-44f8-84e8-95fb3c2d2944 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.123286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6140fd4-111e-44f8-84e8-95fb3c2d2944 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.123979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6140fd4-111e-44f8-84e8-95fb3c2d2944 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.124199 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6140fd4-111e-44f8-84e8-95fb3c2d2944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.125840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6140fd4-111e-44f8-84e8-95fb3c2d2944 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.137135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 921/1846] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:03.141451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6140fd4-111e-44f8-84e8-95fb3c2d2944 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.151562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6140fd4-111e-44f8-84e8-95fb3c2d2944 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.157245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f7699-f5bb-4f6a-99ea-725b59d15cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.157973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f7699-f5bb-4f6a-99ea-725b59d15cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.158272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f7699-f5bb-4f6a-99ea-725b59d15cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.158569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f7699-f5bb-4f6a-99ea-725b59d15cc7 admin admin] PATH_INFO: `/v3/projects/02e4d6f0bc30418dbded397211e4aed3/users/1acffc20d7c64d168ea9cfbca904025b/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.174709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f7699-f5bb-4f6a-99ea-725b59d15cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e4d6f0bc30418dbded397211e4aed3, user_id=1acffc20d7c64d168ea9cfbca904025b, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.177466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f7699-f5bb-4f6a-99ea-725b59d15cc7 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.268140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 922/1847] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:03 2022] PUT /identity/v3/projects/02e4d6f0bc30418dbded397211e4aed3/users/1acffc20d7c64d168ea9cfbca904025b/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:03.277160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19ec9e-fcc1-46a8-84e4-b7443b292f83 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.277482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19ec9e-fcc1-46a8-84e4-b7443b292f83 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.277651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4aab875fbdc4e838891cd8958638160', 'request_id': 'req-faa9a86b-7db7-4f03-8151-8093d900b3cd'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:03.277842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19ec9e-fcc1-46a8-84e4-b7443b292f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.308044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-cd19ec9e-fcc1-46a8-84e4-b7443b292f83 None None] MFA Rules not processed for user `1acffc20d7c64d168ea9cfbca904025b`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:03.309272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cd19ec9e-fcc1-46a8-84e4-b7443b292f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:03.360557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4aab875fbdc4e838891cd8958638160', 'request_id': 'req-faa9a86b-7db7-4f03-8151-8093d900b3cd'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:03.364692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:03.380120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4aab875fbdc4e838891cd8958638160', 'request_id': 'req-faa9a86b-7db7-4f03-8151-8093d900b3cd'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:03.413608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 923/1848] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:03.415624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19ec9e-fcc1-46a8-84e4-b7443b292f83 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.416282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] The token cache is being invalidate because project f4aab875fbdc4e838891cd8958638160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:03.416582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:03.417603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cd19ec9e-fcc1-46a8-84e4-b7443b292f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:03.417772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:03.423532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 926/1849] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:03 2022] DELETE /identity/v3/projects/f4aab875fbdc4e838891cd8958638160 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:03.423532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.423532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:03.489537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd19ec9e-fcc1-46a8-84e4-b7443b292f83 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.495532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.496228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.496535 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.496836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] PATH_INFO: `/v3/projects/118677e72243477e87b5049ba53cb683` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.497535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9a86b-7db7-4f03-8151-8093d900b3cd admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.498089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=118677e72243477e87b5049ba53cb683)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.500076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.502408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a82d9-85db-4c21-8399-e379a16b6ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.503088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a82d9-85db-4c21-8399-e379a16b6ee0 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.503369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a82d9-85db-4c21-8399-e379a16b6ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.508789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a82d9-85db-4c21-8399-e379a16b6ee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.508789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a82d9-85db-4c21-8399-e379a16b6ee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.508789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a82d9-85db-4c21-8399-e379a16b6ee0 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.612339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 927/1850] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:03 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:03.613826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420cc4f-1aab-427b-82b6-cf06f746b799 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.617683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420cc4f-1aab-427b-82b6-cf06f746b799 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.617683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9420cc4f-1aab-427b-82b6-cf06f746b799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.645323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '118677e72243477e87b5049ba53cb683', 'request_id': 'req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:03.655012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9420cc4f-1aab-427b-82b6-cf06f746b799 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:03.656500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9420cc4f-1aab-427b-82b6-cf06f746b799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:03.730585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 928/1851] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:03.732349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420cc4f-1aab-427b-82b6-cf06f746b799 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.748802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9420cc4f-1aab-427b-82b6-cf06f746b799 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.753527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8755d-7e22-4799-acef-c34930e8c209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.754229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8755d-7e22-4799-acef-c34930e8c209 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.754506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8755d-7e22-4799-acef-c34930e8c209 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.754825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8755d-7e22-4799-acef-c34930e8c209 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.755598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8755d-7e22-4799-acef-c34930e8c209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.757495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8755d-7e22-4799-acef-c34930e8c209 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.767599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '118677e72243477e87b5049ba53cb683', 'request_id': 'req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:03.775466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '118677e72243477e87b5049ba53cb683', 'request_id': 'req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:03.786946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] The token cache is being invalidate because project 118677e72243477e87b5049ba53cb683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:03.787232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:03.787916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f7682c77-20d4-4b22-8c0f-3a26e7dd9656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:03.789938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 924/1852] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:03 2022] DELETE /identity/v3/projects/118677e72243477e87b5049ba53cb683 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:03.795347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3c9ff-2971-4ca6-8ed2-390453d9b4b3 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.795634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3c9ff-2971-4ca6-8ed2-390453d9b4b3 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.795901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3c9ff-2971-4ca6-8ed2-390453d9b4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.816013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ddb3c9ff-2971-4ca6-8ed2-390453d9b4b3 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:03.826089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ddb3c9ff-2971-4ca6-8ed2-390453d9b4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:03.826089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 925/1853] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:03.831545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef70e8-7d37-4821-b75d-0a1aa93f34e0 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.831545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef70e8-7d37-4821-b75d-0a1aa93f34e0 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.831545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef70e8-7d37-4821-b75d-0a1aa93f34e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.851061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-dbef70e8-7d37-4821-b75d-0a1aa93f34e0 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:03.852285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-dbef70e8-7d37-4821-b75d-0a1aa93f34e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:03.860584 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 926/1854] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:03.861277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef70e8-7d37-4821-b75d-0a1aa93f34e0 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.871044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:03.873602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef70e8-7d37-4821-b75d-0a1aa93f34e0 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.880155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 929/1855] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:03 2022] POST /identity/v3/users => generated 298 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:03.880155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d25e2-81a1-450c-8132-473f44deed25 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.880412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177b199-e8de-4367-beef-238a0219d152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.880549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d25e2-81a1-450c-8132-473f44deed25 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.880549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d25e2-81a1-450c-8132-473f44deed25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.880721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177b199-e8de-4367-beef-238a0219d152 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.880721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177b199-e8de-4367-beef-238a0219d152 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.880721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177b199-e8de-4367-beef-238a0219d152 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.880721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177b199-e8de-4367-beef-238a0219d152 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:03.881034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177b199-e8de-4367-beef-238a0219d152 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.882408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1177b199-e8de-4367-beef-238a0219d152 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.894780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 927/1856] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:53:03.895683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177b199-e8de-4367-beef-238a0219d152 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.897404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1177b199-e8de-4367-beef-238a0219d152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:03.899369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-3d6d25e2-81a1-450c-8132-473f44deed25 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:03.900478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3d6d25e2-81a1-450c-8132-473f44deed25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:03.908879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 930/1857] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:03.909626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d25e2-81a1-450c-8132-473f44deed25 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.912307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177b199-e8de-4367-beef-238a0219d152 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.918114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eede37-f9d5-4a4a-9d5a-78d279865dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.918592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eede37-f9d5-4a4a-9d5a-78d279865dae admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.919019 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eede37-f9d5-4a4a-9d5a-78d279865dae admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.919294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eede37-f9d5-4a4a-9d5a-78d279865dae admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.919982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eede37-f9d5-4a4a-9d5a-78d279865dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.921563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d25e2-81a1-450c-8132-473f44deed25 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.921679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eede37-f9d5-4a4a-9d5a-78d279865dae admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.926566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a332f4-4148-4fe5-9d14-3b588253b6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.928222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a332f4-4148-4fe5-9d14-3b588253b6d2 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.928222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a332f4-4148-4fe5-9d14-3b588253b6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.928222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a332f4-4148-4fe5-9d14-3b588253b6d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.928499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a332f4-4148-4fe5-9d14-3b588253b6d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.930294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a332f4-4148-4fe5-9d14-3b588253b6d2 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.932413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 928/1858] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:03.933279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eede37-f9d5-4a4a-9d5a-78d279865dae admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.941863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:03.943178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eede37-f9d5-4a4a-9d5a-78d279865dae admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.948080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84e432-a2a0-4d10-88b5-6c9f96d49ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.948666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84e432-a2a0-4d10-88b5-6c9f96d49ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.948945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84e432-a2a0-4d10-88b5-6c9f96d49ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.949228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84e432-a2a0-4d10-88b5-6c9f96d49ed2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.949852 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84e432-a2a0-4d10-88b5-6c9f96d49ed2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:03.950693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84e432-a2a0-4d10-88b5-6c9f96d49ed2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:03.952324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84e432-a2a0-4d10-88b5-6c9f96d49ed2 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:03.963827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 929/1859] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:53:03.964592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84e432-a2a0-4d10-88b5-6c9f96d49ed2 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:03.975914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84e432-a2a0-4d10-88b5-6c9f96d49ed2 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:03.980550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030b9aa-19ce-418f-b737-92556b56bc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:03.981154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030b9aa-19ce-418f-b737-92556b56bc4a admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:03.981432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030b9aa-19ce-418f-b737-92556b56bc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:03.981704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030b9aa-19ce-418f-b737-92556b56bc4a admin admin] PATH_INFO: `/v3/projects/fc54ee5f6c234338ae7344f8d32cb29e/users/e4d0d9788a6e47e6afb12d6bbdc93d8f/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:03.996611 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030b9aa-19ce-418f-b737-92556b56bc4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc54ee5f6c234338ae7344f8d32cb29e, user_id=e4d0d9788a6e47e6afb12d6bbdc93d8f, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.003011 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030b9aa-19ce-418f-b737-92556b56bc4a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.086948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 931/1860] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:03 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:04.087864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a332f4-4148-4fe5-9d14-3b588253b6d2 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.105935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 930/1861] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:03 2022] PUT /identity/v3/projects/fc54ee5f6c234338ae7344f8d32cb29e/users/e4d0d9788a6e47e6afb12d6bbdc93d8f/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:04.106680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030b9aa-19ce-418f-b737-92556b56bc4a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.156749 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a332f4-4148-4fe5-9d14-3b588253b6d2 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.176356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030b9aa-19ce-418f-b737-92556b56bc4a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.182322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6701af7-c8bf-4ba3-a4d1-2761a39d9e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.182927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6701af7-c8bf-4ba3-a4d1-2761a39d9e9d admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.183205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6701af7-c8bf-4ba3-a4d1-2761a39d9e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.183477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6701af7-c8bf-4ba3-a4d1-2761a39d9e9d admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.184098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6701af7-c8bf-4ba3-a4d1-2761a39d9e9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.186561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6701af7-c8bf-4ba3-a4d1-2761a39d9e9d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.228970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b070a-b435-4a80-a7e6-549cf6725b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.229431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b070a-b435-4a80-a7e6-549cf6725b67 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.229889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b070a-b435-4a80-a7e6-549cf6725b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.230373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b070a-b435-4a80-a7e6-549cf6725b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.231412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b070a-b435-4a80-a7e6-549cf6725b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.234597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765b070a-b435-4a80-a7e6-549cf6725b67 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.270076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 931/1862] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:04 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:04.274218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6701af7-c8bf-4ba3-a4d1-2761a39d9e9d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.284576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6701af7-c8bf-4ba3-a4d1-2761a39d9e9d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.290465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8318d64-73e9-4322-bad1-ae23130f4c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.291329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8318d64-73e9-4322-bad1-ae23130f4c07 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.291629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8318d64-73e9-4322-bad1-ae23130f4c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.291941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8318d64-73e9-4322-bad1-ae23130f4c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.292520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8318d64-73e9-4322-bad1-ae23130f4c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.294192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8318d64-73e9-4322-bad1-ae23130f4c07 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.304389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 932/1863] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:04.305371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8318d64-73e9-4322-bad1-ae23130f4c07 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.316148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8318d64-73e9-4322-bad1-ae23130f4c07 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.320963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ef5d9-789b-40b1-b3ca-f5e7eb2b0b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.321864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ef5d9-789b-40b1-b3ca-f5e7eb2b0b2a admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.322270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ef5d9-789b-40b1-b3ca-f5e7eb2b0b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.322664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ef5d9-789b-40b1-b3ca-f5e7eb2b0b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.323310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ef5d9-789b-40b1-b3ca-f5e7eb2b0b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.325573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ef5d9-789b-40b1-b3ca-f5e7eb2b0b2a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.336465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 933/1864] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:04.337297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ef5d9-789b-40b1-b3ca-f5e7eb2b0b2a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.349229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ef5d9-789b-40b1-b3ca-f5e7eb2b0b2a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.355677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e1545-4813-418a-bcd7-489b12b6edd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.356419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e1545-4813-418a-bcd7-489b12b6edd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.356701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e1545-4813-418a-bcd7-489b12b6edd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.356975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e1545-4813-418a-bcd7-489b12b6edd1 admin admin] PATH_INFO: `/v3/projects/fc54ee5f6c234338ae7344f8d32cb29e/users/e4d0d9788a6e47e6afb12d6bbdc93d8f/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.358952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e1545-4813-418a-bcd7-489b12b6edd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc54ee5f6c234338ae7344f8d32cb29e, user_id=e4d0d9788a6e47e6afb12d6bbdc93d8f, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.359168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 932/1865] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:04 2022] POST /identity/v3/projects => generated 371 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:04.360246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b070a-b435-4a80-a7e6-549cf6725b67 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.362072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e1545-4813-418a-bcd7-489b12b6edd1 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.383977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 934/1866] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:04 2022] PUT /identity/v3/projects/fc54ee5f6c234338ae7344f8d32cb29e/users/e4d0d9788a6e47e6afb12d6bbdc93d8f/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:04.384931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e1545-4813-418a-bcd7-489b12b6edd1 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.420776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:04.443312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b070a-b435-4a80-a7e6-549cf6725b67 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.456873 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e1545-4813-418a-bcd7-489b12b6edd1 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.463915 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c2920-0d55-40b0-beee-8444460e34bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.464782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c2920-0d55-40b0-beee-8444460e34bd admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.465108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c2920-0d55-40b0-beee-8444460e34bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.466994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c2920-0d55-40b0-beee-8444460e34bd admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.466994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c2920-0d55-40b0-beee-8444460e34bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.469649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c2920-0d55-40b0-beee-8444460e34bd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.475661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:04.525788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ddc5a-3d8a-469f-b823-2c589af1d71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.530818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ddc5a-3d8a-469f-b823-2c589af1d71a admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.531299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ddc5a-3d8a-469f-b823-2c589af1d71a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.531571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ddc5a-3d8a-469f-b823-2c589af1d71a admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.532239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ddc5a-3d8a-469f-b823-2c589af1d71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.533846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ddc5a-3d8a-469f-b823-2c589af1d71a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.544666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 933/1867] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:04.546354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fc74a-381e-4deb-914c-521f46f21d5d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.546354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fc74a-381e-4deb-914c-521f46f21d5d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.546569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4fc74a-381e-4deb-914c-521f46f21d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.560992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 935/1868] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:04 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:04.562213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c2920-0d55-40b0-beee-8444460e34bd admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.573598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c2920-0d55-40b0-beee-8444460e34bd admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.578968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-de4fc74a-381e-4deb-914c-521f46f21d5d None None] MFA Rules not processed for user `e4d0d9788a6e47e6afb12d6bbdc93d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:04.580156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-de4fc74a-381e-4deb-914c-521f46f21d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:04.585016 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b8acf-7192-4084-baf0-ef063971c9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.586590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b8acf-7192-4084-baf0-ef063971c9d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.586852 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b8acf-7192-4084-baf0-ef063971c9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.587121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b8acf-7192-4084-baf0-ef063971c9d8 admin admin] PATH_INFO: `/v3/projects/2c653555f22946eea6fe8080b4e25bda/users/57e80730bf414ba1ad766603ab13c724/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.605447 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b8acf-7192-4084-baf0-ef063971c9d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c653555f22946eea6fe8080b4e25bda, user_id=57e80730bf414ba1ad766603ab13c724, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.608804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b8acf-7192-4084-baf0-ef063971c9d8 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.659039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 934/1869] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:04.659039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4fc74a-381e-4deb-914c-521f46f21d5d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.669327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4fc74a-381e-4deb-914c-521f46f21d5d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.674009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9584364-0df3-4f5e-b6c8-9f88976d2b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.674575 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9584364-0df3-4f5e-b6c8-9f88976d2b76 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.675185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9584364-0df3-4f5e-b6c8-9f88976d2b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.675185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9584364-0df3-4f5e-b6c8-9f88976d2b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.675847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9584364-0df3-4f5e-b6c8-9f88976d2b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.677487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9584364-0df3-4f5e-b6c8-9f88976d2b76 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.690090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 935/1870] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:04.690090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9584364-0df3-4f5e-b6c8-9f88976d2b76 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.692109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d9584364-0df3-4f5e-b6c8-9f88976d2b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:04.756427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9584364-0df3-4f5e-b6c8-9f88976d2b76 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.760337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e041b-5663-4128-bdb3-fb7d49330be9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.761018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e041b-5663-4128-bdb3-fb7d49330be9 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.761391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e041b-5663-4128-bdb3-fb7d49330be9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.761727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e041b-5663-4128-bdb3-fb7d49330be9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.763577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c68e041b-5663-4128-bdb3-fb7d49330be9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:04.797924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 936/1871] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:04 2022] PUT /identity/v3/projects/2c653555f22946eea6fe8080b4e25bda/users/57e80730bf414ba1ad766603ab13c724/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:04.798798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b8acf-7192-4084-baf0-ef063971c9d8 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.821164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:04.833514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e041b-5663-4128-bdb3-fb7d49330be9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.842790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e041b-5663-4128-bdb3-fb7d49330be9 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.846837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 936/1872] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:04.848103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbedd9-f396-48ea-9e8c-d513285f2b8e None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.848385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbedd9-f396-48ea-9e8c-d513285f2b8e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.848659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cbedd9-f396-48ea-9e8c-d513285f2b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.889492 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f0cbedd9-f396-48ea-9e8c-d513285f2b8e None None] MFA Rules not processed for user `57e80730bf414ba1ad766603ab13c724`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:04.891032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f0cbedd9-f396-48ea-9e8c-d513285f2b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:04.901983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:04.931755 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b8acf-7192-4084-baf0-ef063971c9d8 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.939387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f59d23-5e21-425c-be20-b233f2e5c6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.940121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f59d23-5e21-425c-be20-b233f2e5c6ae admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.940446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f59d23-5e21-425c-be20-b233f2e5c6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.940791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f59d23-5e21-425c-be20-b233f2e5c6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:04.941515 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f59d23-5e21-425c-be20-b233f2e5c6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:04.943739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f59d23-5e21-425c-be20-b233f2e5c6ae admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:04.961981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 937/1873] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:04.964246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f59d23-5e21-425c-be20-b233f2e5c6ae admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.987521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 937/1874] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:04.988439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f59d23-5e21-425c-be20-b233f2e5c6ae admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:04.992242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbedd9-f396-48ea-9e8c-d513285f2b8e None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:04.994862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac97a43-7b11-49ed-a283-ddd049a18c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:04.995842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac97a43-7b11-49ed-a283-ddd049a18c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:04.996274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac97a43-7b11-49ed-a283-ddd049a18c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:04.997037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac97a43-7b11-49ed-a283-ddd049a18c8d admin admin] PATH_INFO: `/v3/projects/17c56bfa9d064876bee1cfe711dfa06c/users/7f6c176c13a54e5a8da7a27fd911f335/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.007144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cbedd9-f396-48ea-9e8c-d513285f2b8e None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:05.012732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee8451-eb14-44d4-96d5-0e4591c624ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:05.013604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8451-eb14-44d4-96d5-0e4591c624ea admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.013941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8451-eb14-44d4-96d5-0e4591c624ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.018598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee8451-eb14-44d4-96d5-0e4591c624ea admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.018598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee8451-eb14-44d4-96d5-0e4591c624ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.018598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee8451-eb14-44d4-96d5-0e4591c624ea admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.020847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac97a43-7b11-49ed-a283-ddd049a18c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c56bfa9d064876bee1cfe711dfa06c, user_id=7f6c176c13a54e5a8da7a27fd911f335, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.025558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac97a43-7b11-49ed-a283-ddd049a18c8d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.122495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 938/1875] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:04 2022] PUT /identity/v3/projects/17c56bfa9d064876bee1cfe711dfa06c/users/7f6c176c13a54e5a8da7a27fd911f335/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:05.129851 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 938/1876] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:04 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:05.131965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac97a43-7b11-49ed-a283-ddd049a18c8d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:05.132497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf06f6e-8ddb-4f33-8382-60119b019610 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.132849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf06f6e-8ddb-4f33-8382-60119b019610 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.133157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf06f6e-8ddb-4f33-8382-60119b019610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.167655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-cdf06f6e-8ddb-4f33-8382-60119b019610 None None] MFA Rules not processed for user `7f6c176c13a54e5a8da7a27fd911f335`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:05.169575 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-cdf06f6e-8ddb-4f33-8382-60119b019610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:05.205052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac97a43-7b11-49ed-a283-ddd049a18c8d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:05.247817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 939/1877] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:05.252491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf06f6e-8ddb-4f33-8382-60119b019610 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:05.258019 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:05.285370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af63c9-42c8-41a6-b169-9b8e622a5df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:05.286016 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af63c9-42c8-41a6-b169-9b8e622a5df3 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.291393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af63c9-42c8-41a6-b169-9b8e622a5df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.291393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af63c9-42c8-41a6-b169-9b8e622a5df3 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.291393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af63c9-42c8-41a6-b169-9b8e622a5df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.291393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af63c9-42c8-41a6-b169-9b8e622a5df3 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.312882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:05.329051 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf06f6e-8ddb-4f33-8382-60119b019610 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:05.333736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926554d-c2a2-4992-9498-bb5ea71e9e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:05.334370 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926554d-c2a2-4992-9498-bb5ea71e9e92 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.334653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926554d-c2a2-4992-9498-bb5ea71e9e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.334927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926554d-c2a2-4992-9498-bb5ea71e9e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.335561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926554d-c2a2-4992-9498-bb5ea71e9e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.337964 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926554d-c2a2-4992-9498-bb5ea71e9e92 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.375584 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 939/1878] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:05 2022] POST /identity/v3/users => generated 298 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:05.379051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af63c9-42c8-41a6-b169-9b8e622a5df3 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:05.396714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af63c9-42c8-41a6-b169-9b8e622a5df3 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:05.402308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adeaa32-60a6-4b9d-b85f-0deaf8758276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:05.403085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adeaa32-60a6-4b9d-b85f-0deaf8758276 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.403761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adeaa32-60a6-4b9d-b85f-0deaf8758276 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.404020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adeaa32-60a6-4b9d-b85f-0deaf8758276 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.404681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adeaa32-60a6-4b9d-b85f-0deaf8758276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.406747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adeaa32-60a6-4b9d-b85f-0deaf8758276 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.423255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 940/1879] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:05.427123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adeaa32-60a6-4b9d-b85f-0deaf8758276 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:05.443089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 940/1880] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:05 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:05.444281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926554d-c2a2-4992-9498-bb5ea71e9e92 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:05.454353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adeaa32-60a6-4b9d-b85f-0deaf8758276 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:05.563229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926554d-c2a2-4992-9498-bb5ea71e9e92 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:05.569347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a9579-036e-4004-a749-51eaa9a23d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:05.569982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a9579-036e-4004-a749-51eaa9a23d43 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.570256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a9579-036e-4004-a749-51eaa9a23d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.570592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a9579-036e-4004-a749-51eaa9a23d43 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.571154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a9579-036e-4004-a749-51eaa9a23d43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.572064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490af9a-ece7-4a20-a3d3-e9e96b234781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:05.572995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490af9a-ece7-4a20-a3d3-e9e96b234781 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.573222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a9579-036e-4004-a749-51eaa9a23d43 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.573343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490af9a-ece7-4a20-a3d3-e9e96b234781 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.573741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490af9a-ece7-4a20-a3d3-e9e96b234781 admin admin] PATH_INFO: `/v3/projects/e9ba9b7e86ec4340b60c6a1071fc205c/users/50bf21eca58844318094690a404a5043/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.592655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490af9a-ece7-4a20-a3d3-e9e96b234781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ba9b7e86ec4340b60c6a1071fc205c, user_id=50bf21eca58844318094690a404a5043, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.595270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490af9a-ece7-4a20-a3d3-e9e96b234781 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.681098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 941/1881] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:05 2022] POST /identity/v3/users => generated 292 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:05.685863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a9579-036e-4004-a749-51eaa9a23d43 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:05.696960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a9579-036e-4004-a749-51eaa9a23d43 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:05.707214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 941/1882] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:05 2022] PUT /identity/v3/projects/e9ba9b7e86ec4340b60c6a1071fc205c/users/50bf21eca58844318094690a404a5043/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:05.718048 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490af9a-ece7-4a20-a3d3-e9e96b234781 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:05.807075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149330d-a379-4c96-937d-3355ee098f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:05.810011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149330d-a379-4c96-937d-3355ee098f0f admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.810764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149330d-a379-4c96-937d-3355ee098f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.811985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149330d-a379-4c96-937d-3355ee098f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.815078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149330d-a379-4c96-937d-3355ee098f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.817601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149330d-a379-4c96-937d-3355ee098f0f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.821031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490af9a-ece7-4a20-a3d3-e9e96b234781 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:05.828200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e36926-9ba4-4f8a-8fe5-b240db857245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:05.829119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e36926-9ba4-4f8a-8fe5-b240db857245 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.829450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e36926-9ba4-4f8a-8fe5-b240db857245 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.832963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e36926-9ba4-4f8a-8fe5-b240db857245 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.832963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e36926-9ba4-4f8a-8fe5-b240db857245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.833888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e36926-9ba4-4f8a-8fe5-b240db857245 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.835581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:05.846678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 942/1883] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:05.847710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 942/1884] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:05.851816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e36926-9ba4-4f8a-8fe5-b240db857245 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:05.852684 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149330d-a379-4c96-937d-3355ee098f0f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:05.876738 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e36926-9ba4-4f8a-8fe5-b240db857245 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:05.878501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149330d-a379-4c96-937d-3355ee098f0f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:05.884880 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3bc9d-6e81-47f9-aacf-c5ec543507df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:05.885067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3415e1-2add-480d-adab-d82148796a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:05.885547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3bc9d-6e81-47f9-aacf-c5ec543507df admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.885824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3bc9d-6e81-47f9-aacf-c5ec543507df admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.885924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3415e1-2add-480d-adab-d82148796a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.886132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3bc9d-6e81-47f9-aacf-c5ec543507df admin admin] PATH_INFO: `/v3/projects/d8f86513120245c280f01c03414a514a/users/9d9d627a992d4190a09cd477aac6136d/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.886288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3415e1-2add-480d-adab-d82148796a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.886652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3415e1-2add-480d-adab-d82148796a55 admin admin] PATH_INFO: `/v3/projects/e9ba9b7e86ec4340b60c6a1071fc205c/users/50bf21eca58844318094690a404a5043/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.889587 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3415e1-2add-480d-adab-d82148796a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ba9b7e86ec4340b60c6a1071fc205c, user_id=50bf21eca58844318094690a404a5043, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.893648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3415e1-2add-480d-adab-d82148796a55 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.905789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3bc9d-6e81-47f9-aacf-c5ec543507df admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f86513120245c280f01c03414a514a, user_id=9d9d627a992d4190a09cd477aac6136d, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:05.909000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb3bc9d-6e81-47f9-aacf-c5ec543507df admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:05.913282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 943/1885] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:05 2022] PUT /identity/v3/projects/e9ba9b7e86ec4340b60c6a1071fc205c/users/50bf21eca58844318094690a404a5043/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:05.921478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b037c-9286-42b6-82e8-ac7680faaf3b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:05.922397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b037c-9286-42b6-82e8-ac7680faaf3b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:05.923558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b037c-9286-42b6-82e8-ac7680faaf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:05.956014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5d4b037c-9286-42b6-82e8-ac7680faaf3b None None] MFA Rules not processed for user `50bf21eca58844318094690a404a5043`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:05.957223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5d4b037c-9286-42b6-82e8-ac7680faaf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:06.042484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 944/1886] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:06.051021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b037c-9286-42b6-82e8-ac7680faaf3b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:06.052569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5d4b037c-9286-42b6-82e8-ac7680faaf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:06.095195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 943/1887] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:05 2022] PUT /identity/v3/projects/d8f86513120245c280f01c03414a514a/users/9d9d627a992d4190a09cd477aac6136d/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:06.108707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3bc9d-6e81-47f9-aacf-c5ec543507df admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:06.123125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b037c-9286-42b6-82e8-ac7680faaf3b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:06.155643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:06.184834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3bc9d-6e81-47f9-aacf-c5ec543507df admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:06.189037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e6f22-be97-44fd-9e36-4af1504539c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:06.189659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d3dcf-9173-4d26-9656-af47169f38f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:06.189829 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e6f22-be97-44fd-9e36-4af1504539c8 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:06.190259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e6f22-be97-44fd-9e36-4af1504539c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:06.190356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d3dcf-9173-4d26-9656-af47169f38f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:06.190610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d3dcf-9173-4d26-9656-af47169f38f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:06.190777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e6f22-be97-44fd-9e36-4af1504539c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:06.190928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d3dcf-9173-4d26-9656-af47169f38f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:06.191759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d3dcf-9173-4d26-9656-af47169f38f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:06.193486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d3dcf-9173-4d26-9656-af47169f38f3 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:06.206578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 944/1888] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:06.211171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d3dcf-9173-4d26-9656-af47169f38f3 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:06.221548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d3dcf-9173-4d26-9656-af47169f38f3 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:06.229566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725152cb-43da-4ad9-86ed-fd6c163c9ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:06.230569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725152cb-43da-4ad9-86ed-fd6c163c9ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:06.234028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725152cb-43da-4ad9-86ed-fd6c163c9ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:06.234028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725152cb-43da-4ad9-86ed-fd6c163c9ba3 admin admin] PATH_INFO: `/v3/projects/d8f86513120245c280f01c03414a514a/users/9d9d627a992d4190a09cd477aac6136d/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:06.234811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725152cb-43da-4ad9-86ed-fd6c163c9ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f86513120245c280f01c03414a514a, user_id=9d9d627a992d4190a09cd477aac6136d, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:06.239526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725152cb-43da-4ad9-86ed-fd6c163c9ba3 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:06.259731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:06.279408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 945/1889] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:06 2022] PUT /identity/v3/projects/d8f86513120245c280f01c03414a514a/users/9d9d627a992d4190a09cd477aac6136d/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:06.290683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253bd56-ddee-4d88-8a15-b37187736876 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:06.291304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253bd56-ddee-4d88-8a15-b37187736876 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:06.291758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253bd56-ddee-4d88-8a15-b37187736876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:06.321190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e6f22-be97-44fd-9e36-4af1504539c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:06.322683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e6f22-be97-44fd-9e36-4af1504539c8 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:06.336630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 945/1890] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:53:06.351190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0253bd56-ddee-4d88-8a15-b37187736876 None None] MFA Rules not processed for user `9d9d627a992d4190a09cd477aac6136d`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:06.352962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0253bd56-ddee-4d88-8a15-b37187736876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:06.475706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 946/1891] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:06.487572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e6f22-be97-44fd-9e36-4af1504539c8 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:06.496191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e6f22-be97-44fd-9e36-4af1504539c8 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:06.549476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c2b3e-d7fb-4d0a-b548-19d82a522a36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:06.550105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c2b3e-d7fb-4d0a-b548-19d82a522a36 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:06.550378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c2b3e-d7fb-4d0a-b548-19d82a522a36 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:06.550643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c2b3e-d7fb-4d0a-b548-19d82a522a36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:06.561125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c2b3e-d7fb-4d0a-b548-19d82a522a36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:06.562698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c2b3e-d7fb-4d0a-b548-19d82a522a36 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:06.575457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 946/1892] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:06.596362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253bd56-ddee-4d88-8a15-b37187736876 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:06.607937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253bd56-ddee-4d88-8a15-b37187736876 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:06.612110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b9f86-c0e6-4e32-8ca1-c6bcc3ec3dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:06.612783 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b9f86-c0e6-4e32-8ca1-c6bcc3ec3dcc service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:06.613121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b9f86-c0e6-4e32-8ca1-c6bcc3ec3dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:06.613616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b9f86-c0e6-4e32-8ca1-c6bcc3ec3dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:06.679084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b9f86-c0e6-4e32-8ca1-c6bcc3ec3dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:06.681189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b9f86-c0e6-4e32-8ca1-c6bcc3ec3dcc service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:06.692132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 947/1893] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:07.625598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c2b3e-d7fb-4d0a-b548-19d82a522a36 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:07.700244 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c2b3e-d7fb-4d0a-b548-19d82a522a36 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:07.706535 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:07.707275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:07.707630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:07.707974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] PATH_INFO: `/v3/users/57e80730bf414ba1ad766603ab13c724` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:07.709269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e80730bf414ba1ad766603ab13c724)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:07.711726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:08.001250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e80730bf414ba1ad766603ab13c724'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:08.055692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b9f86-c0e6-4e32-8ca1-c6bcc3ec3dcc service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:08.067484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b9f86-c0e6-4e32-8ca1-c6bcc3ec3dcc service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:08.075959 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e80730bf414ba1ad766603ab13c724'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:08.079756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:08.080443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:08.080725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:08.084240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] PATH_INFO: `/v3/users/7f6c176c13a54e5a8da7a27fd911f335` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:08.084240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f6c176c13a54e5a8da7a27fd911f335)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:08.088594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:08.091187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e80730bf414ba1ad766603ab13c724'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:08.120341 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:08.123739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e80730bf414ba1ad766603ab13c724'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:08.125719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-63a78600-eb9d-4796-889f-58ab85aca84e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e80730bf414ba1ad766603ab13c724'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:08.147006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 947/1894] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:07 2022] DELETE /identity/v3/users/57e80730bf414ba1ad766603ab13c724 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:08.153358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96633a81-dac7-4dbd-8a23-fe9935858d4a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:08.153358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96633a81-dac7-4dbd-8a23-fe9935858d4a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:08.153358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96633a81-dac7-4dbd-8a23-fe9935858d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:08.197780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-96633a81-dac7-4dbd-8a23-fe9935858d4a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:08.201630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-96633a81-dac7-4dbd-8a23-fe9935858d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:08.284247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 948/1895] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:08.335258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96633a81-dac7-4dbd-8a23-fe9935858d4a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:08.338065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-96633a81-dac7-4dbd-8a23-fe9935858d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:08.448642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:08.463242 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96633a81-dac7-4dbd-8a23-fe9935858d4a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:08.470430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2301a894-653e-4e35-b55b-435ef252dbcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:08.485078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301a894-653e-4e35-b55b-435ef252dbcd service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:08.490063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301a894-653e-4e35-b55b-435ef252dbcd service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:08.491119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2301a894-653e-4e35-b55b-435ef252dbcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:08.500113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f6c176c13a54e5a8da7a27fd911f335'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:08.519725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301a894-653e-4e35-b55b-435ef252dbcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:08.523689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2301a894-653e-4e35-b55b-435ef252dbcd service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:08.540129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 949/1896] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:08.606963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f6c176c13a54e5a8da7a27fd911f335'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:08.627358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f6c176c13a54e5a8da7a27fd911f335'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:08.628119 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f6c176c13a54e5a8da7a27fd911f335'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:08.628748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f6c176c13a54e5a8da7a27fd911f335'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:08.648742 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 948/1897] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:08 2022] DELETE /identity/v3/users/7f6c176c13a54e5a8da7a27fd911f335 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:08.654668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6a4b9-f630-43e2-a90f-193d1ae64c57 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:08.655260 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6a4b9-f630-43e2-a90f-193d1ae64c57 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:08.655413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6a4b9-f630-43e2-a90f-193d1ae64c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:08.682867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a6e6a4b9-f630-43e2-a90f-193d1ae64c57 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:08.685079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a6e6a4b9-f630-43e2-a90f-193d1ae64c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:08.815324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 950/1898] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:08.837775 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:08.839520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:08.929877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4f421-03ae-43d0-a871-e43aa0e5b67a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:08.934163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1bb8d-33a8-481f-926e-8a5294b2826f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:08.935521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1bb8d-33a8-481f-926e-8a5294b2826f service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:08.935890 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1bb8d-33a8-481f-926e-8a5294b2826f service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:08.936162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1bb8d-33a8-481f-926e-8a5294b2826f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:08.953122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1bb8d-33a8-481f-926e-8a5294b2826f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:08.969041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1bb8d-33a8-481f-926e-8a5294b2826f service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:08.996516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 949/1899] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:09.463381 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6a4b9-f630-43e2-a90f-193d1ae64c57 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:09.465911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a6e6a4b9-f630-43e2-a90f-193d1ae64c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:09.491313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6a4b9-f630-43e2-a90f-193d1ae64c57 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:09.497396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:09.497795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:09.498144 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:09.498407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] PATH_INFO: `/v3/projects/2c653555f22946eea6fe8080b4e25bda` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:09.506933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c653555f22946eea6fe8080b4e25bda)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:09.515371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:09.682301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:09.713276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c653555f22946eea6fe8080b4e25bda', 'request_id': 'req-98348ce2-07c3-4d02-ad39-57ab35fa18eb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:09.818875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c653555f22946eea6fe8080b4e25bda', 'request_id': 'req-98348ce2-07c3-4d02-ad39-57ab35fa18eb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:09.828400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c653555f22946eea6fe8080b4e25bda', 'request_id': 'req-98348ce2-07c3-4d02-ad39-57ab35fa18eb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:09.860657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] The token cache is being invalidate because project 2c653555f22946eea6fe8080b4e25bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:09.861013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:09.861837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:09.867869 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 951/1900] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:09 2022] DELETE /identity/v3/projects/2c653555f22946eea6fe8080b4e25bda => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:09.872250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1bb8d-33a8-481f-926e-8a5294b2826f service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:09.886175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1bb8d-33a8-481f-926e-8a5294b2826f service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:10.015265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:10.015265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:10.015265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:10.015265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] PATH_INFO: `/v3/users/50bf21eca58844318094690a404a5043` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:10.015908 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] RBAC: Authorizing `identity:delete_user(user_id=50bf21eca58844318094690a404a5043)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:10.017618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:10.240075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:10.243055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:10.261056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50bf21eca58844318094690a404a5043'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:10.274267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98348ce2-07c3-4d02-ad39-57ab35fa18eb admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:10.287035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:10.291701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:10.291701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:10.291701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] PATH_INFO: `/v3/projects/17c56bfa9d064876bee1cfe711dfa06c` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:10.291701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c56bfa9d064876bee1cfe711dfa06c)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:10.293084 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:10.367952 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50bf21eca58844318094690a404a5043'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:10.385048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:10.407781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50bf21eca58844318094690a404a5043'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:10.408254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50bf21eca58844318094690a404a5043'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:10.409289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50bf21eca58844318094690a404a5043'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:10.445463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 950/1901] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:09 2022] DELETE /identity/v3/users/50bf21eca58844318094690a404a5043 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:10.633452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c56bfa9d064876bee1cfe711dfa06c', 'request_id': 'req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:10.744668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c56bfa9d064876bee1cfe711dfa06c', 'request_id': 'req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:10.762812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c56bfa9d064876bee1cfe711dfa06c', 'request_id': 'req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:10.784102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] The token cache is being invalidate because project 17c56bfa9d064876bee1cfe711dfa06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:10.784398 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:10.799623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:10.799623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 952/1902] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:10 2022] DELETE /identity/v3/projects/17c56bfa9d064876bee1cfe711dfa06c => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:10.801011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:10.802649 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:10.902371 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c1ccd-17cd-473d-8905-4981fb6faaad admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:10.995257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:10.997115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:10.997416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:10.997708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] PATH_INFO: `/v3/users/9d9d627a992d4190a09cd477aac6136d` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:10.998965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9d627a992d4190a09cd477aac6136d)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:11.002755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:11.195466 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:11.197996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:11.226348 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbf0e4-f219-44f0-aeaa-eb753d24e1ee admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:11.241323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9d627a992d4190a09cd477aac6136d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:11.243246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:11.244559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:11.247956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:11.247956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] PATH_INFO: `/v3/projects/e9ba9b7e86ec4340b60c6a1071fc205c` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:11.250072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ba9b7e86ec4340b60c6a1071fc205c)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:11.253223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:11.320402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9d627a992d4190a09cd477aac6136d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:11.339687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9d627a992d4190a09cd477aac6136d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:11.341040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9d627a992d4190a09cd477aac6136d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:11.341815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d64fc9a0-81bd-47ee-aa0e-41de7b94a3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9d627a992d4190a09cd477aac6136d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:11.371025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 951/1903] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:10 2022] DELETE /identity/v3/users/9d9d627a992d4190a09cd477aac6136d => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:11.390693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ba9b7e86ec4340b60c6a1071fc205c', 'request_id': 'req-aaad18f0-129b-40f6-a85a-e6cd6e220666'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:11.533039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ba9b7e86ec4340b60c6a1071fc205c', 'request_id': 'req-aaad18f0-129b-40f6-a85a-e6cd6e220666'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:11.543057 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ba9b7e86ec4340b60c6a1071fc205c', 'request_id': 'req-aaad18f0-129b-40f6-a85a-e6cd6e220666'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:11.555998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] The token cache is being invalidate because project e9ba9b7e86ec4340b60c6a1071fc205c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:11.556295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:11.556957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-aaad18f0-129b-40f6-a85a-e6cd6e220666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:11.558539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 953/1904] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:11 2022] DELETE /identity/v3/projects/e9ba9b7e86ec4340b60c6a1071fc205c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:11.564506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2c921-d8e1-42f0-b523-a286d1d0a222 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:11.565595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2c921-d8e1-42f0-b523-a286d1d0a222 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:11.566302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2c921-d8e1-42f0-b523-a286d1d0a222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:11.577875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:11.599741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6fc2c921-d8e1-42f0-b523-a286d1d0a222 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:11.600950 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6fc2c921-d8e1-42f0-b523-a286d1d0a222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:11.688192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 952/1905] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:11.696080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd3639-c014-4918-836a-7d2e16e50398 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:11.696361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd3639-c014-4918-836a-7d2e16e50398 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:11.696675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd3639-c014-4918-836a-7d2e16e50398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:11.719259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-fdcd3639-c014-4918-836a-7d2e16e50398 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:11.734433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-fdcd3639-c014-4918-836a-7d2e16e50398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:11.745140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 954/1906] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:11.749780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2c921-d8e1-42f0-b523-a286d1d0a222 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:11.767729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2c921-d8e1-42f0-b523-a286d1d0a222 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:11.785838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e251ba-2464-47ae-afcd-effd37439a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:11.786486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e251ba-2464-47ae-afcd-effd37439a0f admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:11.786841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e251ba-2464-47ae-afcd-effd37439a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:11.787017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e251ba-2464-47ae-afcd-effd37439a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:11.787666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e251ba-2464-47ae-afcd-effd37439a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:11.787977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e251ba-2464-47ae-afcd-effd37439a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:11.789549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e251ba-2464-47ae-afcd-effd37439a0f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:11.805744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 953/1907] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:53:11.810455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd3639-c014-4918-836a-7d2e16e50398 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:11.821887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd3639-c014-4918-836a-7d2e16e50398 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:11.829845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9a813-dd54-4c09-abbb-7b156d490afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:11.830550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9a813-dd54-4c09-abbb-7b156d490afa admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:11.830925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9a813-dd54-4c09-abbb-7b156d490afa admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:11.831350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9a813-dd54-4c09-abbb-7b156d490afa admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:11.832150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9a813-dd54-4c09-abbb-7b156d490afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:11.834527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9a813-dd54-4c09-abbb-7b156d490afa admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:11.873514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e251ba-2464-47ae-afcd-effd37439a0f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:11.875796 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a4e251ba-2464-47ae-afcd-effd37439a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:11.904188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e251ba-2464-47ae-afcd-effd37439a0f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:11.917399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:11.917576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:11.917834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:11.918587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] PATH_INFO: `/v3/projects/d8f86513120245c280f01c03414a514a` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:11.919913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f86513120245c280f01c03414a514a)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:11.922418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:11.929323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=19,keystone:SELECT=77,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:11.937334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 955/1908] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:11 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:11.949938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9a813-dd54-4c09-abbb-7b156d490afa admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.036754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f86513120245c280f01c03414a514a', 'request_id': 'req-711849fd-5622-4ef2-aa47-3df01c9fae1b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:12.046824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9a813-dd54-4c09-abbb-7b156d490afa admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.110561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f86513120245c280f01c03414a514a', 'request_id': 'req-711849fd-5622-4ef2-aa47-3df01c9fae1b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:12.126028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f86513120245c280f01c03414a514a', 'request_id': 'req-711849fd-5622-4ef2-aa47-3df01c9fae1b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:12.133106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079b3ed-b826-4958-9bc3-34bda2d2c2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.133762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079b3ed-b826-4958-9bc3-34bda2d2c2df admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.134130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079b3ed-b826-4958-9bc3-34bda2d2c2df admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.134404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079b3ed-b826-4958-9bc3-34bda2d2c2df admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.135035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079b3ed-b826-4958-9bc3-34bda2d2c2df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.137649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0079b3ed-b826-4958-9bc3-34bda2d2c2df admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.138224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] The token cache is being invalidate because project d8f86513120245c280f01c03414a514a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:12.138511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:12.145420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-711849fd-5622-4ef2-aa47-3df01c9fae1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:12.145420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 954/1909] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:11 2022] DELETE /identity/v3/projects/d8f86513120245c280f01c03414a514a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:12.145759 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02369437-5264-4d31-a4fe-d55d8115abe5 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.146160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02369437-5264-4d31-a4fe-d55d8115abe5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.146349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02369437-5264-4d31-a4fe-d55d8115abe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.166083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-02369437-5264-4d31-a4fe-d55d8115abe5 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:12.167400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-02369437-5264-4d31-a4fe-d55d8115abe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:12.175748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 955/1910] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:12.183194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55663b7-eb08-4420-b20a-14343c586c4e None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.183479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55663b7-eb08-4420-b20a-14343c586c4e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.184273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55663b7-eb08-4420-b20a-14343c586c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.211538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d55663b7-eb08-4420-b20a-14343c586c4e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:12.212789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d55663b7-eb08-4420-b20a-14343c586c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:12.220956 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 956/1911] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:12.225456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55663b7-eb08-4420-b20a-14343c586c4e None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.231159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 956/1912] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:11 2022] POST /identity/v3/users => generated 288 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:12.235409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079b3ed-b826-4958-9bc3-34bda2d2c2df admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.236127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55663b7-eb08-4420-b20a-14343c586c4e None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.237237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0079b3ed-b826-4958-9bc3-34bda2d2c2df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:12.240962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9bfc0-16b1-4643-ac72-e60040e8f9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.241742 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9bfc0-16b1-4643-ac72-e60040e8f9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.241864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9bfc0-16b1-4643-ac72-e60040e8f9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.242186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9bfc0-16b1-4643-ac72-e60040e8f9b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.242834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9bfc0-16b1-4643-ac72-e60040e8f9b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:12.243131 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9bfc0-16b1-4643-ac72-e60040e8f9b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.244824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9bfc0-16b1-4643-ac72-e60040e8f9b3 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.249924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0079b3ed-b826-4958-9bc3-34bda2d2c2df admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.255156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf1cf4-51b1-481a-b1a6-ca4185bb09aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.255755 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf1cf4-51b1-481a-b1a6-ca4185bb09aa admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.256032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf1cf4-51b1-481a-b1a6-ca4185bb09aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.256434 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf1cf4-51b1-481a-b1a6-ca4185bb09aa admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.257112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf1cf4-51b1-481a-b1a6-ca4185bb09aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.257495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 957/1913] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:53:12.258732 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebf1cf4-51b1-481a-b1a6-ca4185bb09aa admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.264780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9bfc0-16b1-4643-ac72-e60040e8f9b3 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.271152 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 957/1914] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:12.274279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf1cf4-51b1-481a-b1a6-ca4185bb09aa admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.275863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9bfc0-16b1-4643-ac72-e60040e8f9b3 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.281094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523a7bd-22e1-49c2-a74d-6410e06008c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.281910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523a7bd-22e1-49c2-a74d-6410e06008c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.282234 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523a7bd-22e1-49c2-a74d-6410e06008c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.282560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523a7bd-22e1-49c2-a74d-6410e06008c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.283226 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523a7bd-22e1-49c2-a74d-6410e06008c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.285471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523a7bd-22e1-49c2-a74d-6410e06008c9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.285901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf1cf4-51b1-481a-b1a6-ca4185bb09aa admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.295228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef8833-a296-435a-87d6-439b74228074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.295228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef8833-a296-435a-87d6-439b74228074 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.295228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef8833-a296-435a-87d6-439b74228074 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.295228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef8833-a296-435a-87d6-439b74228074 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.296027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef8833-a296-435a-87d6-439b74228074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.296027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef8833-a296-435a-87d6-439b74228074 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.306082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 958/1915] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:12.313605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef8833-a296-435a-87d6-439b74228074 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.324983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef8833-a296-435a-87d6-439b74228074 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.329763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65797b21-c98a-40f3-9dcd-9816afb5d4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.330534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797b21-c98a-40f3-9dcd-9816afb5d4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.330809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797b21-c98a-40f3-9dcd-9816afb5d4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.331082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797b21-c98a-40f3-9dcd-9816afb5d4c1 admin admin] PATH_INFO: `/v3/projects/0af26346147e439e9ff887051a9efdca/users/cae5360ab4aa40a89088bc5790685ad1/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.347019 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65797b21-c98a-40f3-9dcd-9816afb5d4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af26346147e439e9ff887051a9efdca, user_id=cae5360ab4aa40a89088bc5790685ad1, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.349868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65797b21-c98a-40f3-9dcd-9816afb5d4c1 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.403223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 958/1916] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:12 2022] POST /identity/v3/projects => generated 389 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:12.407416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523a7bd-22e1-49c2-a74d-6410e06008c9 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.438243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:12.463604 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 959/1917] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:12 2022] PUT /identity/v3/projects/0af26346147e439e9ff887051a9efdca/users/cae5360ab4aa40a89088bc5790685ad1/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:12.474473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f527c-4fc8-4952-9d8b-90dae993a3c9 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.474964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f527c-4fc8-4952-9d8b-90dae993a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.475236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f527c-4fc8-4952-9d8b-90dae993a3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.490643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523a7bd-22e1-49c2-a74d-6410e06008c9 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.521681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8d5f527c-4fc8-4952-9d8b-90dae993a3c9 None None] MFA Rules not processed for user `cae5360ab4aa40a89088bc5790685ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:12.523151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8d5f527c-4fc8-4952-9d8b-90dae993a3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:12.553225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a474f7c-e09d-4462-84a9-7d5ba32fb699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.553935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a474f7c-e09d-4462-84a9-7d5ba32fb699 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.554236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a474f7c-e09d-4462-84a9-7d5ba32fb699 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.554503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a474f7c-e09d-4462-84a9-7d5ba32fb699 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.555121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a474f7c-e09d-4462-84a9-7d5ba32fb699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.557016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a474f7c-e09d-4462-84a9-7d5ba32fb699 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.591216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 960/1918] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:12.601091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f527c-4fc8-4952-9d8b-90dae993a3c9 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.606330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f527c-4fc8-4952-9d8b-90dae993a3c9 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.610901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf436f0f-f007-44b4-9f81-ed7be23b6e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.611543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf436f0f-f007-44b4-9f81-ed7be23b6e11 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.611816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf436f0f-f007-44b4-9f81-ed7be23b6e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.612086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf436f0f-f007-44b4-9f81-ed7be23b6e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.612696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf436f0f-f007-44b4-9f81-ed7be23b6e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.614630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf436f0f-f007-44b4-9f81-ed7be23b6e11 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.635865 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:12.689659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 959/1919] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:12 2022] POST /identity/v3/users => generated 295 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:12.693720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a474f7c-e09d-4462-84a9-7d5ba32fb699 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.704139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a474f7c-e09d-4462-84a9-7d5ba32fb699 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.708804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4760a-00b2-4079-84ce-63518e20ff91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.709393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4760a-00b2-4079-84ce-63518e20ff91 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.709667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4760a-00b2-4079-84ce-63518e20ff91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.709933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de4760a-00b2-4079-84ce-63518e20ff91 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.710651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4760a-00b2-4079-84ce-63518e20ff91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.712254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de4760a-00b2-4079-84ce-63518e20ff91 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.722774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 960/1920] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:12.726649 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4760a-00b2-4079-84ce-63518e20ff91 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.737757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de4760a-00b2-4079-84ce-63518e20ff91 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.744350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 961/1921] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:12 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:12.744476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48301613-c2d6-46bb-a5ac-60ad19d1c289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.744591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301613-c2d6-46bb-a5ac-60ad19d1c289 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.744866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301613-c2d6-46bb-a5ac-60ad19d1c289 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.745143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301613-c2d6-46bb-a5ac-60ad19d1c289 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.745805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48301613-c2d6-46bb-a5ac-60ad19d1c289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.747664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf436f0f-f007-44b4-9f81-ed7be23b6e11 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.748192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48301613-c2d6-46bb-a5ac-60ad19d1c289 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.762206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 961/1922] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:12.763354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48301613-c2d6-46bb-a5ac-60ad19d1c289 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.773016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48301613-c2d6-46bb-a5ac-60ad19d1c289 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.814402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf436f0f-f007-44b4-9f81-ed7be23b6e11 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.819562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69450169-9225-4c98-bfa8-b4d311a34013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.820163 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69450169-9225-4c98-bfa8-b4d311a34013 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.820441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69450169-9225-4c98-bfa8-b4d311a34013 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.820711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69450169-9225-4c98-bfa8-b4d311a34013 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.821397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69450169-9225-4c98-bfa8-b4d311a34013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.823406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69450169-9225-4c98-bfa8-b4d311a34013 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.830276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2ed8f-cd2e-4a79-a462-91ab57182176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.831018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2ed8f-cd2e-4a79-a462-91ab57182176 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.831295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2ed8f-cd2e-4a79-a462-91ab57182176 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.831564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2ed8f-cd2e-4a79-a462-91ab57182176 admin admin] PATH_INFO: `/v3/projects/8148db36d85e4352b93052d4fbe456b5/users/c9377f03cae0421d86dcc7acec10b673/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.846878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2ed8f-cd2e-4a79-a462-91ab57182176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8148db36d85e4352b93052d4fbe456b5, user_id=c9377f03cae0421d86dcc7acec10b673, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.849688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2ed8f-cd2e-4a79-a462-91ab57182176 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.905841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 962/1923] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:12 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:12.909740 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69450169-9225-4c98-bfa8-b4d311a34013 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.920218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69450169-9225-4c98-bfa8-b4d311a34013 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.926744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e5e2f-08fe-4200-b38b-7bd6d985a169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:12.927341 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e5e2f-08fe-4200-b38b-7bd6d985a169 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.927620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e5e2f-08fe-4200-b38b-7bd6d985a169 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.927893 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e5e2f-08fe-4200-b38b-7bd6d985a169 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.928532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e5e2f-08fe-4200-b38b-7bd6d985a169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:12.931005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e5e2f-08fe-4200-b38b-7bd6d985a169 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:12.936016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 962/1924] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:12 2022] PUT /identity/v3/projects/8148db36d85e4352b93052d4fbe456b5/users/c9377f03cae0421d86dcc7acec10b673/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:12.944162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 963/1925] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:12.947128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a856127-444a-4a12-91fb-154e18eb4f91 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:12.947409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a856127-444a-4a12-91fb-154e18eb4f91 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:12.947682 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a856127-444a-4a12-91fb-154e18eb4f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:12.953315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e5e2f-08fe-4200-b38b-7bd6d985a169 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:12.959037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e5e2f-08fe-4200-b38b-7bd6d985a169 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:12.975702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-5a856127-444a-4a12-91fb-154e18eb4f91 None None] MFA Rules not processed for user `c9377f03cae0421d86dcc7acec10b673`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:12.978727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5a856127-444a-4a12-91fb-154e18eb4f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:13.024303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cb28b-b69d-4143-b7bd-eea7e9eaa7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.025257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cb28b-b69d-4143-b7bd-eea7e9eaa7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.025531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cb28b-b69d-4143-b7bd-eea7e9eaa7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.025799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cb28b-b69d-4143-b7bd-eea7e9eaa7ac admin admin] PATH_INFO: `/v3/projects/0e8eda8067c9412a936ca767061dafd9/users/32ba5659587c4c8b96484230b8710823/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.042710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cb28b-b69d-4143-b7bd-eea7e9eaa7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8eda8067c9412a936ca767061dafd9, user_id=32ba5659587c4c8b96484230b8710823, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.046070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2cb28b-b69d-4143-b7bd-eea7e9eaa7ac admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.050241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 963/1926] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:13.057138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a856127-444a-4a12-91fb-154e18eb4f91 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:13.067999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a856127-444a-4a12-91fb-154e18eb4f91 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:13.073105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10597c03-3f49-42a9-acc2-bfbb9782d6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.074090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10597c03-3f49-42a9-acc2-bfbb9782d6bf admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.074090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10597c03-3f49-42a9-acc2-bfbb9782d6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.074313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10597c03-3f49-42a9-acc2-bfbb9782d6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.077533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10597c03-3f49-42a9-acc2-bfbb9782d6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.077533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10597c03-3f49-42a9-acc2-bfbb9782d6bf admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.130726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 964/1927] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:12 2022] PUT /identity/v3/projects/0e8eda8067c9412a936ca767061dafd9/users/32ba5659587c4c8b96484230b8710823/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:13.134924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cb28b-b69d-4143-b7bd-eea7e9eaa7ac admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:13.161109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:13.180107 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:13.187633 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 964/1928] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:13 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:13.191471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10597c03-3f49-42a9-acc2-bfbb9782d6bf admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:13.198823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cb28b-b69d-4143-b7bd-eea7e9eaa7ac admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:13.257602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007dba-645c-49cc-bd1d-dc8054e1c239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.258251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007dba-645c-49cc-bd1d-dc8054e1c239 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.258529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007dba-645c-49cc-bd1d-dc8054e1c239 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.258800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007dba-645c-49cc-bd1d-dc8054e1c239 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.259462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18007dba-645c-49cc-bd1d-dc8054e1c239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.259767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10597c03-3f49-42a9-acc2-bfbb9782d6bf admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:13.261047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18007dba-645c-49cc-bd1d-dc8054e1c239 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.264614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558bfe6-e375-470c-8f97-60ddb0721dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.265217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558bfe6-e375-470c-8f97-60ddb0721dea admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.265496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558bfe6-e375-470c-8f97-60ddb0721dea admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.265768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558bfe6-e375-470c-8f97-60ddb0721dea admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.266419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558bfe6-e375-470c-8f97-60ddb0721dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.268299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558bfe6-e375-470c-8f97-60ddb0721dea admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.270931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 965/1929] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:13.275175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007dba-645c-49cc-bd1d-dc8054e1c239 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:13.285006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007dba-645c-49cc-bd1d-dc8054e1c239 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:13.290109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c87a75-c0a7-4bec-ae9e-e1518e8dd28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.290789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c87a75-c0a7-4bec-ae9e-e1518e8dd28e admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.291056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c87a75-c0a7-4bec-ae9e-e1518e8dd28e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.293971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c87a75-c0a7-4bec-ae9e-e1518e8dd28e admin admin] PATH_INFO: `/v3/projects/0e8eda8067c9412a936ca767061dafd9/users/32ba5659587c4c8b96484230b8710823/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.293971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c87a75-c0a7-4bec-ae9e-e1518e8dd28e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e8eda8067c9412a936ca767061dafd9, user_id=32ba5659587c4c8b96484230b8710823, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.296403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c87a75-c0a7-4bec-ae9e-e1518e8dd28e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.306485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 966/1930] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:13 2022] PUT /identity/v3/projects/0e8eda8067c9412a936ca767061dafd9/users/32ba5659587c4c8b96484230b8710823/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:13.317125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087314-494d-46e6-9afc-1f9b16c47747 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.322785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087314-494d-46e6-9afc-1f9b16c47747 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.322785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087314-494d-46e6-9afc-1f9b16c47747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.344846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-e7087314-494d-46e6-9afc-1f9b16c47747 None None] MFA Rules not processed for user `32ba5659587c4c8b96484230b8710823`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:13.346291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e7087314-494d-46e6-9afc-1f9b16c47747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:13.373351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 965/1931] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:13 2022] POST /identity/v3/users => generated 300 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:13.377278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558bfe6-e375-470c-8f97-60ddb0721dea admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:13.387112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558bfe6-e375-470c-8f97-60ddb0721dea admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:13.421405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 967/1932] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:13.431313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7087314-494d-46e6-9afc-1f9b16c47747 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:13.432926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e7087314-494d-46e6-9afc-1f9b16c47747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:13.449017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8aa3de-766f-4464-b8b1-385455e7971c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.453328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8aa3de-766f-4464-b8b1-385455e7971c admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.453328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8aa3de-766f-4464-b8b1-385455e7971c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.453328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8aa3de-766f-4464-b8b1-385455e7971c admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.453328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8aa3de-766f-4464-b8b1-385455e7971c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.453328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8aa3de-766f-4464-b8b1-385455e7971c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.462230 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 966/1933] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:13.466478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8aa3de-766f-4464-b8b1-385455e7971c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:13.474724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:13.482097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8aa3de-766f-4464-b8b1-385455e7971c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:13.482943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8aa8a-2114-4166-ab7a-28218513d2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.483611 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8aa8a-2114-4166-ab7a-28218513d2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.483886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8aa8a-2114-4166-ab7a-28218513d2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.484160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8aa8a-2114-4166-ab7a-28218513d2b7 admin admin] PATH_INFO: `/v3/projects/40245838625146798eec90e71e5cd8ae/users/722acf3336134975b03da1e58f2cbd7e/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.492944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7087314-494d-46e6-9afc-1f9b16c47747 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:13.497468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72919d82-ad20-46f5-bafb-b69d110847c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.497915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:13.498050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72919d82-ad20-46f5-bafb-b69d110847c7 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.498257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72919d82-ad20-46f5-bafb-b69d110847c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.498550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72919d82-ad20-46f5-bafb-b69d110847c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.501071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8aa8a-2114-4166-ab7a-28218513d2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40245838625146798eec90e71e5cd8ae, user_id=722acf3336134975b03da1e58f2cbd7e, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.504044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8aa8a-2114-4166-ab7a-28218513d2b7 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.573831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72919d82-ad20-46f5-bafb-b69d110847c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.575858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72919d82-ad20-46f5-bafb-b69d110847c7 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.585640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 968/1934] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:13.630354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 967/1935] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:13 2022] PUT /identity/v3/projects/40245838625146798eec90e71e5cd8ae/users/722acf3336134975b03da1e58f2cbd7e/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:13.634610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72919d82-ad20-46f5-bafb-b69d110847c7 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:13.646115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72919d82-ad20-46f5-bafb-b69d110847c7 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:13.704309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600baca-1e38-4b1f-ab20-c97313babc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.704922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600baca-1e38-4b1f-ab20-c97313babc7b admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.705264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600baca-1e38-4b1f-ab20-c97313babc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.705601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600baca-1e38-4b1f-ab20-c97313babc7b admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.706322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600baca-1e38-4b1f-ab20-c97313babc7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.707913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600baca-1e38-4b1f-ab20-c97313babc7b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.718378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 969/1936] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:13.722555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8aa8a-2114-4166-ab7a-28218513d2b7 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:13.732909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8aa8a-2114-4166-ab7a-28218513d2b7 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:13.737605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0eaed1-9e97-4eb1-a2bb-70b30b8ad0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.738316 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eaed1-9e97-4eb1-a2bb-70b30b8ad0df admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.738593 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eaed1-9e97-4eb1-a2bb-70b30b8ad0df admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.738889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0eaed1-9e97-4eb1-a2bb-70b30b8ad0df admin admin] PATH_INFO: `/v3/projects/40245838625146798eec90e71e5cd8ae/users/722acf3336134975b03da1e58f2cbd7e/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.740862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0eaed1-9e97-4eb1-a2bb-70b30b8ad0df admin admin] RBAC: Authorizing `identity:create_grant(project_id=40245838625146798eec90e71e5cd8ae, user_id=722acf3336134975b03da1e58f2cbd7e, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.743523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0eaed1-9e97-4eb1-a2bb-70b30b8ad0df admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.754326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 968/1937] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:13 2022] PUT /identity/v3/projects/40245838625146798eec90e71e5cd8ae/users/722acf3336134975b03da1e58f2cbd7e/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:13.766147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28127f3e-96ec-4f83-9dac-b909b97c2995 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.766642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28127f3e-96ec-4f83-9dac-b909b97c2995 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.766750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28127f3e-96ec-4f83-9dac-b909b97c2995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.795219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-28127f3e-96ec-4f83-9dac-b909b97c2995 None None] MFA Rules not processed for user `722acf3336134975b03da1e58f2cbd7e`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:13.796428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-28127f3e-96ec-4f83-9dac-b909b97c2995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:13.866134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 970/1938] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:13.877696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0eaed1-9e97-4eb1-a2bb-70b30b8ad0df admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:13.879425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fb0eaed1-9e97-4eb1-a2bb-70b30b8ad0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:13.940217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0eaed1-9e97-4eb1-a2bb-70b30b8ad0df admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:13.944073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934f26c-ea9d-40b4-a1e2-4ad7e27eafa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:13.944670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934f26c-ea9d-40b4-a1e2-4ad7e27eafa4 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:13.944904 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934f26c-ea9d-40b4-a1e2-4ad7e27eafa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:13.945173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934f26c-ea9d-40b4-a1e2-4ad7e27eafa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:13.955325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934f26c-ea9d-40b4-a1e2-4ad7e27eafa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:13.956670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934f26c-ea9d-40b4-a1e2-4ad7e27eafa4 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:13.966333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 969/1939] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:14.104107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28127f3e-96ec-4f83-9dac-b909b97c2995 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:14.116983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:14.181972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28127f3e-96ec-4f83-9dac-b909b97c2995 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:14.197134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:14.197824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:14.198137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:14.198408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] PATH_INFO: `/v3/users/c9377f03cae0421d86dcc7acec10b673` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:14.199704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9377f03cae0421d86dcc7acec10b673)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:14.202483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:14.266694 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934f26c-ea9d-40b4-a1e2-4ad7e27eafa4 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:14.275158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934f26c-ea9d-40b4-a1e2-4ad7e27eafa4 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:14.278976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b9d5f-be5f-4131-8730-4ea10f4acd6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:14.279674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b9d5f-be5f-4131-8730-4ea10f4acd6e service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:14.279831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b9d5f-be5f-4131-8730-4ea10f4acd6e service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:14.280095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b9d5f-be5f-4131-8730-4ea10f4acd6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:14.281836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d70b9d5f-be5f-4131-8730-4ea10f4acd6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:14.363909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b9d5f-be5f-4131-8730-4ea10f4acd6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:14.365381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b9d5f-be5f-4131-8730-4ea10f4acd6e service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:14.374975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 970/1940] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:14.452749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9377f03cae0421d86dcc7acec10b673'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:14.549652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9377f03cae0421d86dcc7acec10b673'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:14.557231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9377f03cae0421d86dcc7acec10b673'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:14.569236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9377f03cae0421d86dcc7acec10b673'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:14.569925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9377f03cae0421d86dcc7acec10b673'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:14.576273 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 971/1941] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:14 2022] DELETE /identity/v3/users/c9377f03cae0421d86dcc7acec10b673 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:14.580796 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b06e-1ea1-4137-9846-a8ccc632fec9 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:14.581080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b06e-1ea1-4137-9846-a8ccc632fec9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:14.581352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486b06e-1ea1-4137-9846-a8ccc632fec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:14.601746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f486b06e-1ea1-4137-9846-a8ccc632fec9 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:14.602983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f486b06e-1ea1-4137-9846-a8ccc632fec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:14.665685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 971/1942] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:14.676041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:14.677473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:14.735114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e15df-3c89-4e2e-ab96-6df8790cdc15 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:14.738804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a29d38-ce41-4e5a-aeff-059f74694074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:14.739401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a29d38-ce41-4e5a-aeff-059f74694074 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:14.739677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a29d38-ce41-4e5a-aeff-059f74694074 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:14.739946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a29d38-ce41-4e5a-aeff-059f74694074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:14.748895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a29d38-ce41-4e5a-aeff-059f74694074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:14.750349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a29d38-ce41-4e5a-aeff-059f74694074 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:14.759536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 972/1943] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:15.362970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b06e-1ea1-4137-9846-a8ccc632fec9 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:15.364498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f486b06e-1ea1-4137-9846-a8ccc632fec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:15.377751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486b06e-1ea1-4137-9846-a8ccc632fec9 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:15.382418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:15.383061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:15.384489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:15.384489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] PATH_INFO: `/v3/projects/8148db36d85e4352b93052d4fbe456b5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:15.384764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8148db36d85e4352b93052d4fbe456b5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:15.386710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:15.402205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:15.548125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8148db36d85e4352b93052d4fbe456b5', 'request_id': 'req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:15.612582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8148db36d85e4352b93052d4fbe456b5', 'request_id': 'req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:15.619286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8148db36d85e4352b93052d4fbe456b5', 'request_id': 'req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:15.630564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] The token cache is being invalidate because project 8148db36d85e4352b93052d4fbe456b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:15.630863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:15.631514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:15.633181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 972/1944] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:15 2022] DELETE /identity/v3/projects/8148db36d85e4352b93052d4fbe456b5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:15.637946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a29d38-ce41-4e5a-aeff-059f74694074 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:15.647072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a29d38-ce41-4e5a-aeff-059f74694074 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:15.677333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:15.705001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:15.705617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:15.705892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:15.706252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] PATH_INFO: `/v3/users/722acf3336134975b03da1e58f2cbd7e` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:15.707387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=722acf3336134975b03da1e58f2cbd7e)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:15.709766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:15.893987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '722acf3336134975b03da1e58f2cbd7e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:15.967155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '722acf3336134975b03da1e58f2cbd7e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:15.974730 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '722acf3336134975b03da1e58f2cbd7e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:15.987248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '722acf3336134975b03da1e58f2cbd7e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:15.987922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-963816d3-c590-4885-a226-8856a0cb72a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '722acf3336134975b03da1e58f2cbd7e'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:15.993872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 973/1945] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:15 2022] DELETE /identity/v3/users/722acf3336134975b03da1e58f2cbd7e => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:16.512506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:16.514248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:16.531757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfa38e-d2c3-48e1-827f-b3a9e249ce96 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:16.595463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:16.596125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:16.596401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:16.596669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] PATH_INFO: `/v3/projects/40245838625146798eec90e71e5cd8ae` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:16.597810 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40245838625146798eec90e71e5cd8ae)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:16.600432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:16.738369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40245838625146798eec90e71e5cd8ae', 'request_id': 'req-5854383b-ebbd-4fc5-a11a-9c6952e2d237'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:16.858468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40245838625146798eec90e71e5cd8ae', 'request_id': 'req-5854383b-ebbd-4fc5-a11a-9c6952e2d237'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:16.865365 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40245838625146798eec90e71e5cd8ae', 'request_id': 'req-5854383b-ebbd-4fc5-a11a-9c6952e2d237'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:16.876725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] The token cache is being invalidate because project 40245838625146798eec90e71e5cd8ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:16.877003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:16.877635 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5854383b-ebbd-4fc5-a11a-9c6952e2d237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:16.879132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 973/1946] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:16 2022] DELETE /identity/v3/projects/40245838625146798eec90e71e5cd8ae => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:16.885742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca0c4a-0a1c-4cb2-bbe2-d8062ba1c8ae None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:16.886295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca0c4a-0a1c-4cb2-bbe2-d8062ba1c8ae None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:16.886385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca0c4a-0a1c-4cb2-bbe2-d8062ba1c8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:16.908082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c9ca0c4a-0a1c-4cb2-bbe2-d8062ba1c8ae None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:16.909594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c9ca0c4a-0a1c-4cb2-bbe2-d8062ba1c8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:16.971274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 974/1947] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:16.979119 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a80bd-1c8c-4dd7-8239-20bb9d15fbd3 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:16.979405 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a80bd-1c8c-4dd7-8239-20bb9d15fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:16.979676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a80bd-1c8c-4dd7-8239-20bb9d15fbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.000489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-895a80bd-1c8c-4dd7-8239-20bb9d15fbd3 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:17.017353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-895a80bd-1c8c-4dd7-8239-20bb9d15fbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:17.017353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 974/1948] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:17.020235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca0c4a-0a1c-4cb2-bbe2-d8062ba1c8ae None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:17.029774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca0c4a-0a1c-4cb2-bbe2-d8062ba1c8ae None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:17.034653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c9f09-67b2-40f2-b492-de1ffb70da3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:17.035255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c9f09-67b2-40f2-b492-de1ffb70da3c admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:17.035527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c9f09-67b2-40f2-b492-de1ffb70da3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:17.035796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c9f09-67b2-40f2-b492-de1ffb70da3c admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.036441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c9f09-67b2-40f2-b492-de1ffb70da3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:17.036741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c9f09-67b2-40f2-b492-de1ffb70da3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:17.038369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c9f09-67b2-40f2-b492-de1ffb70da3c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:17.050402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 975/1949] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:53:17.054933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a80bd-1c8c-4dd7-8239-20bb9d15fbd3 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:17.064638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a80bd-1c8c-4dd7-8239-20bb9d15fbd3 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:17.069269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f91e-8b61-4148-84e9-b674b6476b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:17.069860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f91e-8b61-4148-84e9-b674b6476b78 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:17.070164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f91e-8b61-4148-84e9-b674b6476b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:17.070799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96f91e-8b61-4148-84e9-b674b6476b78 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.071116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96f91e-8b61-4148-84e9-b674b6476b78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:17.073020 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96f91e-8b61-4148-84e9-b674b6476b78 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:17.196644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:17.204079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 975/1950] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:17 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:17.208332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c9f09-67b2-40f2-b492-de1ffb70da3c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:17.218269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c9f09-67b2-40f2-b492-de1ffb70da3c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:17.256679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:17.274830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db43033-1088-4575-a975-5b6ac5ea59a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:17.275421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db43033-1088-4575-a975-5b6ac5ea59a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:17.275693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db43033-1088-4575-a975-5b6ac5ea59a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:17.275958 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db43033-1088-4575-a975-5b6ac5ea59a1 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.276579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db43033-1088-4575-a975-5b6ac5ea59a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:17.278615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db43033-1088-4575-a975-5b6ac5ea59a1 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:17.365779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 976/1951] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:17 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:17.368483 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f91e-8b61-4148-84e9-b674b6476b78 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:17.380141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96f91e-8b61-4148-84e9-b674b6476b78 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:17.385156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc8da1-b731-49ed-8ef0-e722506c61cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:17.386666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8da1-b731-49ed-8ef0-e722506c61cb admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:17.386666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8da1-b731-49ed-8ef0-e722506c61cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:17.386666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc8da1-b731-49ed-8ef0-e722506c61cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.387060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc8da1-b731-49ed-8ef0-e722506c61cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:17.388677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc8da1-b731-49ed-8ef0-e722506c61cb admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:17.399104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 976/1952] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:17.403082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db43033-1088-4575-a975-5b6ac5ea59a1 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:17.413722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db43033-1088-4575-a975-5b6ac5ea59a1 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:17.419350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc310fb8-f58e-40eb-ad2f-1ee3304ac66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:17.420001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc310fb8-f58e-40eb-ad2f-1ee3304ac66c admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:17.420277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc310fb8-f58e-40eb-ad2f-1ee3304ac66c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:17.420955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc310fb8-f58e-40eb-ad2f-1ee3304ac66c admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.421254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc310fb8-f58e-40eb-ad2f-1ee3304ac66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:17.422951 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc310fb8-f58e-40eb-ad2f-1ee3304ac66c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:17.434495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 977/1953] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:17.439117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc8da1-b731-49ed-8ef0-e722506c61cb admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:17.449479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc8da1-b731-49ed-8ef0-e722506c61cb admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:17.453871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942abc3-f424-4318-9d67-231a325e52fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:17.454561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942abc3-f424-4318-9d67-231a325e52fb admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:17.454837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942abc3-f424-4318-9d67-231a325e52fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:17.455109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942abc3-f424-4318-9d67-231a325e52fb admin admin] PATH_INFO: `/v3/projects/da5754d2703946199f1c80de9dd50735/users/0473b4e8d8b241d4ad64d23c20dfc0e1/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.471175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942abc3-f424-4318-9d67-231a325e52fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5754d2703946199f1c80de9dd50735, user_id=0473b4e8d8b241d4ad64d23c20dfc0e1, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:17.473868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7942abc3-f424-4318-9d67-231a325e52fb admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:17.601386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 977/1954] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:17 2022] PUT /identity/v3/projects/da5754d2703946199f1c80de9dd50735/users/0473b4e8d8b241d4ad64d23c20dfc0e1/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:17.612547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c660-8dd9-48f3-b8d2-aa5968a6cb18 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:17.612831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c660-8dd9-48f3-b8d2-aa5968a6cb18 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:17.613102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69c660-8dd9-48f3-b8d2-aa5968a6cb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.641015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-0c69c660-8dd9-48f3-b8d2-aa5968a6cb18 None None] MFA Rules not processed for user `0473b4e8d8b241d4ad64d23c20dfc0e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:17.642421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0c69c660-8dd9-48f3-b8d2-aa5968a6cb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:17.707803 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 978/1955] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:17.720315 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942abc3-f424-4318-9d67-231a325e52fb admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:17.722303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-7942abc3-f424-4318-9d67-231a325e52fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:17.781273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7942abc3-f424-4318-9d67-231a325e52fb admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:17.789631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7de52-8c7b-4cbb-ba2e-0115d5783833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:17.789631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7de52-8c7b-4cbb-ba2e-0115d5783833 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:17.789631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7de52-8c7b-4cbb-ba2e-0115d5783833 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:17.789631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7de52-8c7b-4cbb-ba2e-0115d5783833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.797916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7de52-8c7b-4cbb-ba2e-0115d5783833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:17.799432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7de52-8c7b-4cbb-ba2e-0115d5783833 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:17.808415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c660-8dd9-48f3-b8d2-aa5968a6cb18 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:17.810121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0c69c660-8dd9-48f3-b8d2-aa5968a6cb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:17.818339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 978/1956] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:17.853693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7de52-8c7b-4cbb-ba2e-0115d5783833 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:17.914824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69c660-8dd9-48f3-b8d2-aa5968a6cb18 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:17.924096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:17.924992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:17.925427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:17.925854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] PATH_INFO: `/v3/users/cae5360ab4aa40a89088bc5790685ad1` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.927629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae5360ab4aa40a89088bc5790685ad1)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:17.930323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:17.964279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7de52-8c7b-4cbb-ba2e-0115d5783833 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:17.969768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:17.970417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:17.970697 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:17.970967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] PATH_INFO: `/v3/users/0473b4e8d8b241d4ad64d23c20dfc0e1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:17.972096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=0473b4e8d8b241d4ad64d23c20dfc0e1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:17.974261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:18.187031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae5360ab4aa40a89088bc5790685ad1'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:18.187249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0473b4e8d8b241d4ad64d23c20dfc0e1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:18.191714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:18.275065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae5360ab4aa40a89088bc5790685ad1'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:18.283298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae5360ab4aa40a89088bc5790685ad1'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:18.295326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae5360ab4aa40a89088bc5790685ad1'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:18.295995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-af8084f0-063a-4c2d-8aef-a350f5812c32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae5360ab4aa40a89088bc5790685ad1'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:18.302036 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 979/1957] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:17 2022] DELETE /identity/v3/users/cae5360ab4aa40a89088bc5790685ad1 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:18.306482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc40e0-8240-48d9-a5d4-9ed19877cf58 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:18.306802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc40e0-8240-48d9-a5d4-9ed19877cf58 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:18.306978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc40e0-8240-48d9-a5d4-9ed19877cf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:18.327090 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-17fc40e0-8240-48d9-a5d4-9ed19877cf58 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:18.327808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-17fc40e0-8240-48d9-a5d4-9ed19877cf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:18.384511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 980/1958] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:18.394707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc40e0-8240-48d9-a5d4-9ed19877cf58 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:18.394960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0473b4e8d8b241d4ad64d23c20dfc0e1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:18.397001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-17fc40e0-8240-48d9-a5d4-9ed19877cf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:18.408455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:18.410200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0473b4e8d8b241d4ad64d23c20dfc0e1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:18.410923 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0473b4e8d8b241d4ad64d23c20dfc0e1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:18.411520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b6608cce-face-4ec9-9f4d-833709c551ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0473b4e8d8b241d4ad64d23c20dfc0e1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:18.423887 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 979/1959] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:17 2022] DELETE /identity/v3/users/0473b4e8d8b241d4ad64d23c20dfc0e1 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:18.428463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02273b00-5c43-4140-80a4-da01fa523109 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:18.428741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02273b00-5c43-4140-80a4-da01fa523109 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:18.429005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02273b00-5c43-4140-80a4-da01fa523109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:18.447878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-02273b00-5c43-4140-80a4-da01fa523109 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:18.449074 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-02273b00-5c43-4140-80a4-da01fa523109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:18.455319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc40e0-8240-48d9-a5d4-9ed19877cf58 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:18.506754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f34fe2-ff05-414f-bc99-5b4ef3dcda38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:18.506754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f34fe2-ff05-414f-bc99-5b4ef3dcda38 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:18.506754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f34fe2-ff05-414f-bc99-5b4ef3dcda38 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:18.506754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f34fe2-ff05-414f-bc99-5b4ef3dcda38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:18.508343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-31f34fe2-ff05-414f-bc99-5b4ef3dcda38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:18.514867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 980/1960] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:18.524550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02273b00-5c43-4140-80a4-da01fa523109 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:18.524824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f34fe2-ff05-414f-bc99-5b4ef3dcda38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:18.528554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-02273b00-5c43-4140-80a4-da01fa523109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:18.528668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f34fe2-ff05-414f-bc99-5b4ef3dcda38 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:18.539614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02273b00-5c43-4140-80a4-da01fa523109 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:18.539957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 981/1961] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:53:18.543424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fa653-5112-4875-af07-faa1b40f6f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:18.544030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fa653-5112-4875-af07-faa1b40f6f36 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:18.544302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fa653-5112-4875-af07-faa1b40f6f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:18.544570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fa653-5112-4875-af07-faa1b40f6f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:18.554210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fa653-5112-4875-af07-faa1b40f6f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:18.555701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fa653-5112-4875-af07-faa1b40f6f36 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:18.566975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 981/1962] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:53:19.110762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f34fe2-ff05-414f-bc99-5b4ef3dcda38 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:19.112285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-31f34fe2-ff05-414f-bc99-5b4ef3dcda38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:19.126223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f34fe2-ff05-414f-bc99-5b4ef3dcda38 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:19.132617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:19.133237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:19.133513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:19.133782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] PATH_INFO: `/v3/projects/0af26346147e439e9ff887051a9efdca` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:19.140219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af26346147e439e9ff887051a9efdca)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:19.140219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:19.177887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:19.191586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fa653-5112-4875-af07-faa1b40f6f36 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:19.193866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bc3fa653-5112-4875-af07-faa1b40f6f36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:19.218454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fa653-5112-4875-af07-faa1b40f6f36 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:19.225608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:19.226437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:19.226778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:19.227173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] PATH_INFO: `/v3/projects/da5754d2703946199f1c80de9dd50735` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:19.231517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5754d2703946199f1c80de9dd50735)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:19.233994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:19.314528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af26346147e439e9ff887051a9efdca', 'request_id': 'req-88be707f-a9f6-4a62-98ef-b9dfe01511af'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.348447 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5754d2703946199f1c80de9dd50735', 'request_id': 'req-bdb99dbd-8523-4145-acfe-12da3fab7243'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.401022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af26346147e439e9ff887051a9efdca', 'request_id': 'req-88be707f-a9f6-4a62-98ef-b9dfe01511af'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.408230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af26346147e439e9ff887051a9efdca', 'request_id': 'req-88be707f-a9f6-4a62-98ef-b9dfe01511af'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.419378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] The token cache is being invalidate because project 0af26346147e439e9ff887051a9efdca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:19.419641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.420287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.421811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 982/1963] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:19 2022] DELETE /identity/v3/projects/0af26346147e439e9ff887051a9efdca => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:19.425845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:19.426979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:19.501324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be707f-a9f6-4a62-98ef-b9dfe01511af admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:19.521071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5754d2703946199f1c80de9dd50735', 'request_id': 'req-bdb99dbd-8523-4145-acfe-12da3fab7243'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.533420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5754d2703946199f1c80de9dd50735', 'request_id': 'req-bdb99dbd-8523-4145-acfe-12da3fab7243'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.553868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] The token cache is being invalidate because project da5754d2703946199f1c80de9dd50735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:19.554451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.555919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-bdb99dbd-8523-4145-acfe-12da3fab7243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.558954 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 982/1964] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:19 2022] DELETE /identity/v3/projects/da5754d2703946199f1c80de9dd50735 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:19.564884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b749c-5d5b-42c7-a570-2ce02cc96a8c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:19.565370 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b749c-5d5b-42c7-a570-2ce02cc96a8c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:19.565850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b749c-5d5b-42c7-a570-2ce02cc96a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:19.576547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:19.577181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:19.577456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:19.577725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] PATH_INFO: `/v3/users/32ba5659587c4c8b96484230b8710823` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:19.578923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] RBAC: Authorizing `identity:delete_user(user_id=32ba5659587c4c8b96484230b8710823)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:19.580878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:19.596376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-507b749c-5d5b-42c7-a570-2ce02cc96a8c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:19.597587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-507b749c-5d5b-42c7-a570-2ce02cc96a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:19.608174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 983/1965] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:19.617047 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25115bed-a862-4d5f-991f-eb3e0886c19c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:19.617404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25115bed-a862-4d5f-991f-eb3e0886c19c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:19.617750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25115bed-a862-4d5f-991f-eb3e0886c19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:19.638984 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-25115bed-a862-4d5f-991f-eb3e0886c19c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:19.640231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-25115bed-a862-4d5f-991f-eb3e0886c19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:19.648502 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 984/1966] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:19.653481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25115bed-a862-4d5f-991f-eb3e0886c19c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:19.663938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25115bed-a862-4d5f-991f-eb3e0886c19c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:19.669332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3dfba-8659-455c-8616-1b77e7e945a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:19.669933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3dfba-8659-455c-8616-1b77e7e945a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:19.670235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3dfba-8659-455c-8616-1b77e7e945a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:19.670504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3dfba-8659-455c-8616-1b77e7e945a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:19.671165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3dfba-8659-455c-8616-1b77e7e945a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:19.671469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3dfba-8659-455c-8616-1b77e7e945a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:19.673265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa3dfba-8659-455c-8616-1b77e7e945a6 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:19.688530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 985/1967] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:53:19.689779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3dfba-8659-455c-8616-1b77e7e945a6 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:19.700331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3dfba-8659-455c-8616-1b77e7e945a6 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:19.705540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7322a-7f84-4398-b6d5-426715d7653b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:19.706232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7322a-7f84-4398-b6d5-426715d7653b admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:19.706549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7322a-7f84-4398-b6d5-426715d7653b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:19.706869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7322a-7f84-4398-b6d5-426715d7653b admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:19.707555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7322a-7f84-4398-b6d5-426715d7653b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:19.709595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7322a-7f84-4398-b6d5-426715d7653b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:19.808132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32ba5659587c4c8b96484230b8710823'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.834065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 986/1968] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:19 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:19.837641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7322a-7f84-4398-b6d5-426715d7653b admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:19.848386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:19.897912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32ba5659587c4c8b96484230b8710823'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.903125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7322a-7f84-4398-b6d5-426715d7653b admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:19.904322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:19.905160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3494f3-0aa1-4e00-967e-49d838eef1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:19.905823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3494f3-0aa1-4e00-967e-49d838eef1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:19.906130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3494f3-0aa1-4e00-967e-49d838eef1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:19.906454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3494f3-0aa1-4e00-967e-49d838eef1f4 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:19.907013 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3494f3-0aa1-4e00-967e-49d838eef1f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:19.908345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32ba5659587c4c8b96484230b8710823'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.909271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3494f3-0aa1-4e00-967e-49d838eef1f4 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:19.922777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32ba5659587c4c8b96484230b8710823'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.923572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32ba5659587c4c8b96484230b8710823'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:19.929757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 983/1969] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:19 2022] DELETE /identity/v3/users/32ba5659587c4c8b96484230b8710823 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:20.043093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 987/1970] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:19 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:20.047762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:20.050201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:20.118215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c53a27-61d9-4401-83ab-a2dff754404b admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:20.123494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eabb41-0e0d-4d57-81ca-e4ec1a68d043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:20.124094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eabb41-0e0d-4d57-81ca-e4ec1a68d043 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:20.124365 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eabb41-0e0d-4d57-81ca-e4ec1a68d043 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:20.124636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eabb41-0e0d-4d57-81ca-e4ec1a68d043 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:20.125370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eabb41-0e0d-4d57-81ca-e4ec1a68d043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:20.127012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eabb41-0e0d-4d57-81ca-e4ec1a68d043 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:20.147582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 984/1971] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:20.148092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3494f3-0aa1-4e00-967e-49d838eef1f4 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:20.160726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3494f3-0aa1-4e00-967e-49d838eef1f4 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:20.166079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b599a2-8005-4bde-abb2-d5ddb43cca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:20.178558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b599a2-8005-4bde-abb2-d5ddb43cca0f admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:20.178558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b599a2-8005-4bde-abb2-d5ddb43cca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:20.178558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b599a2-8005-4bde-abb2-d5ddb43cca0f admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:20.178558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b599a2-8005-4bde-abb2-d5ddb43cca0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:20.178558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b599a2-8005-4bde-abb2-d5ddb43cca0f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:20.209896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eabb41-0e0d-4d57-81ca-e4ec1a68d043 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:20.215415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 988/1972] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:20.222609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eabb41-0e0d-4d57-81ca-e4ec1a68d043 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:20.230194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc02722-1b6b-4b56-8610-0920e45f5a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:20.231724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc02722-1b6b-4b56-8610-0920e45f5a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:20.231724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc02722-1b6b-4b56-8610-0920e45f5a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:20.231991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc02722-1b6b-4b56-8610-0920e45f5a96 admin admin] PATH_INFO: `/v3/projects/9dba2ee6e64e4766a0d021b4552a0b2c/users/8d77f31d797448b6ad4964fa96e2d492/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:20.256564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc02722-1b6b-4b56-8610-0920e45f5a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dba2ee6e64e4766a0d021b4552a0b2c, user_id=8d77f31d797448b6ad4964fa96e2d492, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:20.262635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc02722-1b6b-4b56-8610-0920e45f5a96 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:20.423701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 985/1973] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:20 2022] PUT /identity/v3/projects/9dba2ee6e64e4766a0d021b4552a0b2c/users/8d77f31d797448b6ad4964fa96e2d492/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:20.428784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20d67-4a40-4666-b8ba-a58ed2d34bb5 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:20.429075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20d67-4a40-4666-b8ba-a58ed2d34bb5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:20.429343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f20d67-4a40-4666-b8ba-a58ed2d34bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:20.450283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d4f20d67-4a40-4666-b8ba-a58ed2d34bb5 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:20.451639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d4f20d67-4a40-4666-b8ba-a58ed2d34bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:20.512859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 989/1974] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:20.534907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc02722-1b6b-4b56-8610-0920e45f5a96 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:20.536792 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6bc02722-1b6b-4b56-8610-0920e45f5a96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:20.604638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc02722-1b6b-4b56-8610-0920e45f5a96 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:20.608457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888a0d9-eb27-44aa-b1c5-a5c970175944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:20.609151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888a0d9-eb27-44aa-b1c5-a5c970175944 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:20.609436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888a0d9-eb27-44aa-b1c5-a5c970175944 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:20.609823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888a0d9-eb27-44aa-b1c5-a5c970175944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:20.619680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888a0d9-eb27-44aa-b1c5-a5c970175944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:20.621243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888a0d9-eb27-44aa-b1c5-a5c970175944 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:20.631963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 986/1975] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:20.797837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f20d67-4a40-4666-b8ba-a58ed2d34bb5 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:20.800242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d4f20d67-4a40-4666-b8ba-a58ed2d34bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:20.818325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f20d67-4a40-4666-b8ba-a58ed2d34bb5 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:20.841368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:20.841679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:20.841979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:20.843083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] PATH_INFO: `/v3/projects/0e8eda8067c9412a936ca767061dafd9` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:20.844636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e8eda8067c9412a936ca767061dafd9)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:20.847273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:20.988373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e8eda8067c9412a936ca767061dafd9', 'request_id': 'req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:21.046771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e8eda8067c9412a936ca767061dafd9', 'request_id': 'req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:21.055627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e8eda8067c9412a936ca767061dafd9', 'request_id': 'req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:21.066965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] The token cache is being invalidate because project 0e8eda8067c9412a936ca767061dafd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:21.067262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:21.067943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b554d0c5-460e-4c9b-b2e6-02952c5eb2e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:21.069563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 990/1976] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:20 2022] DELETE /identity/v3/projects/0e8eda8067c9412a936ca767061dafd9 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:21.075376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24de38-5ed5-4784-ba47-faf80a68e424 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.075863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24de38-5ed5-4784-ba47-faf80a68e424 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.084752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f24de38-5ed5-4784-ba47-faf80a68e424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:21.095217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-2f24de38-5ed5-4784-ba47-faf80a68e424 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:21.096487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2f24de38-5ed5-4784-ba47-faf80a68e424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:21.153611 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 987/1977] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:21.161638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854d4e9-cb45-47d5-9738-1d32bf027677 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.161924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854d4e9-cb45-47d5-9738-1d32bf027677 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.162227 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854d4e9-cb45-47d5-9738-1d32bf027677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:21.183921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9854d4e9-cb45-47d5-9738-1d32bf027677 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:21.185247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9854d4e9-cb45-47d5-9738-1d32bf027677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:21.193836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 991/1978] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:21.198254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24de38-5ed5-4784-ba47-faf80a68e424 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:21.208691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f24de38-5ed5-4784-ba47-faf80a68e424 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:21.214378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9e3cc-fb23-4374-b6a9-db3b03131f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:21.216517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9e3cc-fb23-4374-b6a9-db3b03131f6c admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.216517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9e3cc-fb23-4374-b6a9-db3b03131f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.216517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9e3cc-fb23-4374-b6a9-db3b03131f6c admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:21.216517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9e3cc-fb23-4374-b6a9-db3b03131f6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:21.216517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9e3cc-fb23-4374-b6a9-db3b03131f6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:21.218123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9e3cc-fb23-4374-b6a9-db3b03131f6c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:21.234318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 988/1979] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:53:21.235903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854d4e9-cb45-47d5-9738-1d32bf027677 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:21.242122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:21.245782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9854d4e9-cb45-47d5-9738-1d32bf027677 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:21.251234 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56c836-eecb-4c94-b9b0-761dae1a6ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:21.251846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56c836-eecb-4c94-b9b0-761dae1a6ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.252126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56c836-eecb-4c94-b9b0-761dae1a6ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.252397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56c836-eecb-4c94-b9b0-761dae1a6ba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:21.253019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56c836-eecb-4c94-b9b0-761dae1a6ba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:21.255923 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b56c836-eecb-4c94-b9b0-761dae1a6ba1 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:21.291544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:21.385533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 992/1980] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:21 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:21.389782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9e3cc-fb23-4374-b6a9-db3b03131f6c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:21.399863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9e3cc-fb23-4374-b6a9-db3b03131f6c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:21.455775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a778f1-dd0e-4a29-8344-690cca09c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:21.456385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a778f1-dd0e-4a29-8344-690cca09c3a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.456659 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a778f1-dd0e-4a29-8344-690cca09c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.456927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a778f1-dd0e-4a29-8344-690cca09c3a4 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:21.457603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a778f1-dd0e-4a29-8344-690cca09c3a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:21.459563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a778f1-dd0e-4a29-8344-690cca09c3a4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:21.575354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 989/1981] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:21 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:21.576861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56c836-eecb-4c94-b9b0-761dae1a6ba1 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:21.589470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56c836-eecb-4c94-b9b0-761dae1a6ba1 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:21.595034 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294ae5c-c8fd-4b49-b1aa-4f6e80e6fd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:21.599605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294ae5c-c8fd-4b49-b1aa-4f6e80e6fd4f admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.599605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294ae5c-c8fd-4b49-b1aa-4f6e80e6fd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.599605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294ae5c-c8fd-4b49-b1aa-4f6e80e6fd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:21.599605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294ae5c-c8fd-4b49-b1aa-4f6e80e6fd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:21.599605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294ae5c-c8fd-4b49-b1aa-4f6e80e6fd4f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:21.609722 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 993/1982] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:21.613928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a778f1-dd0e-4a29-8344-690cca09c3a4 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:21.624902 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a778f1-dd0e-4a29-8344-690cca09c3a4 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:21.629991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06892ff-6306-49e0-8969-e85947ca0a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:21.634118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06892ff-6306-49e0-8969-e85947ca0a10 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.634118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06892ff-6306-49e0-8969-e85947ca0a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.634118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06892ff-6306-49e0-8969-e85947ca0a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:21.634118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06892ff-6306-49e0-8969-e85947ca0a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:21.634118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06892ff-6306-49e0-8969-e85947ca0a10 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:21.648578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 990/1983] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:21.649779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294ae5c-c8fd-4b49-b1aa-4f6e80e6fd4f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:21.660571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294ae5c-c8fd-4b49-b1aa-4f6e80e6fd4f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:21.665869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355ebf2-a001-4b82-b7fc-21d2fa3c6c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:21.668574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355ebf2-a001-4b82-b7fc-21d2fa3c6c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.668574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355ebf2-a001-4b82-b7fc-21d2fa3c6c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.668574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355ebf2-a001-4b82-b7fc-21d2fa3c6c03 admin admin] PATH_INFO: `/v3/projects/004336e4f4f347918a221ef96db56a93/users/3903ea3c41734facbb54bf671f03ec97/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:21.683312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355ebf2-a001-4b82-b7fc-21d2fa3c6c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=004336e4f4f347918a221ef96db56a93, user_id=3903ea3c41734facbb54bf671f03ec97, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:21.686092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4355ebf2-a001-4b82-b7fc-21d2fa3c6c03 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:21.773652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 994/1984] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:21 2022] PUT /identity/v3/projects/004336e4f4f347918a221ef96db56a93/users/3903ea3c41734facbb54bf671f03ec97/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:21.785995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03ebdf-5e02-4137-ace4-a2f670091e1c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.786516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03ebdf-5e02-4137-ace4-a2f670091e1c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.786981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03ebdf-5e02-4137-ace4-a2f670091e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:21.824252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5c03ebdf-5e02-4137-ace4-a2f670091e1c None None] MFA Rules not processed for user `3903ea3c41734facbb54bf671f03ec97`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:21.825505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5c03ebdf-5e02-4137-ace4-a2f670091e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:21.894313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355ebf2-a001-4b82-b7fc-21d2fa3c6c03 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:21.896111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4355ebf2-a001-4b82-b7fc-21d2fa3c6c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:21.906746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 991/1985] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:21.919776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03ebdf-5e02-4137-ace4-a2f670091e1c None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:21.921297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5c03ebdf-5e02-4137-ace4-a2f670091e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:21.976152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355ebf2-a001-4b82-b7fc-21d2fa3c6c03 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:21.981711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:21.982631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.983513 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.983513 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] PATH_INFO: `/v3/users/1acffc20d7c64d168ea9cfbca904025b` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:21.984484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1acffc20d7c64d168ea9cfbca904025b)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:21.986520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:21.991264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03ebdf-5e02-4137-ace4-a2f670091e1c None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:21.995576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776609a-8d0b-40c6-a41e-b4a050dc95fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:21.995903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776609a-8d0b-40c6-a41e-b4a050dc95fd service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:21.996180 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776609a-8d0b-40c6-a41e-b4a050dc95fd service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:21.996451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776609a-8d0b-40c6-a41e-b4a050dc95fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:22.005215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776609a-8d0b-40c6-a41e-b4a050dc95fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:22.006732 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776609a-8d0b-40c6-a41e-b4a050dc95fd service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:22.017593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 992/1986] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:53:22.217977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1acffc20d7c64d168ea9cfbca904025b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:22.343805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1acffc20d7c64d168ea9cfbca904025b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:22.359304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1acffc20d7c64d168ea9cfbca904025b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:22.360540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1acffc20d7c64d168ea9cfbca904025b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:22.361341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1acffc20d7c64d168ea9cfbca904025b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:22.374474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 995/1987] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:21 2022] DELETE /identity/v3/users/1acffc20d7c64d168ea9cfbca904025b => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:22.380130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88918838-71d7-42b6-b81a-986b6e6e66ea None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:22.381062 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88918838-71d7-42b6-b81a-986b6e6e66ea None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:22.381643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88918838-71d7-42b6-b81a-986b6e6e66ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:22.403914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-88918838-71d7-42b6-b81a-986b6e6e66ea None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:22.405156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-88918838-71d7-42b6-b81a-986b6e6e66ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:22.478374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 993/1988] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:22.488677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:22.490137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:22.548676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b808a-86f4-4c82-b80e-969405c8dde1 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:22.553044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d927a-ad3a-410e-92d9-93c3c15fff51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:22.553672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d927a-ad3a-410e-92d9-93c3c15fff51 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:22.553946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d927a-ad3a-410e-92d9-93c3c15fff51 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:22.554238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d927a-ad3a-410e-92d9-93c3c15fff51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:22.563621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d927a-ad3a-410e-92d9-93c3c15fff51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:22.565048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d927a-ad3a-410e-92d9-93c3c15fff51 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:22.576205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 996/1989] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:22.894234 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:23.052384 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:23.669491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88918838-71d7-42b6-b81a-986b6e6e66ea None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:23.670080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-88918838-71d7-42b6-b81a-986b6e6e66ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:23.681905 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88918838-71d7-42b6-b81a-986b6e6e66ea None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:23.687552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:23.688262 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:23.688582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:23.688849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] PATH_INFO: `/v3/projects/02e4d6f0bc30418dbded397211e4aed3` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:23.690066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] RBAC: Authorizing `identity:delete_project(project_id=02e4d6f0bc30418dbded397211e4aed3)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:23.692034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:23.844592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02e4d6f0bc30418dbded397211e4aed3', 'request_id': 'req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:23.951274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02e4d6f0bc30418dbded397211e4aed3', 'request_id': 'req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:23.961307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02e4d6f0bc30418dbded397211e4aed3', 'request_id': 'req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:23.985014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] The token cache is being invalidate because project 02e4d6f0bc30418dbded397211e4aed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:23.985548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:23.986436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:23.996052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 994/1990] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:23 2022] DELETE /identity/v3/projects/02e4d6f0bc30418dbded397211e4aed3 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:23.996192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d927a-ad3a-410e-92d9-93c3c15fff51 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:24.004655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d927a-ad3a-410e-92d9-93c3c15fff51 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:24.071521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:24.072391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:24.076518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:24.076518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] PATH_INFO: `/v3/users/e4d0d9788a6e47e6afb12d6bbdc93d8f` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:24.076518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d0d9788a6e47e6afb12d6bbdc93d8f)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:24.076833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:24.315786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d0d9788a6e47e6afb12d6bbdc93d8f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:24.376466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d0d9788a6e47e6afb12d6bbdc93d8f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:24.392259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d0d9788a6e47e6afb12d6bbdc93d8f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:24.393863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d0d9788a6e47e6afb12d6bbdc93d8f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:24.394580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c0254403-2e09-4e02-bd86-f5e77dc33995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d0d9788a6e47e6afb12d6bbdc93d8f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:24.424717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 997/1991] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:23 2022] DELETE /identity/v3/users/e4d0d9788a6e47e6afb12d6bbdc93d8f => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:25.325130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:25.327489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:25.351700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e948d0-c8ab-46fc-9e8c-762a7b5fd90a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:25.446628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:25.447669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:25.448128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:25.448597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] PATH_INFO: `/v3/projects/fc54ee5f6c234338ae7344f8d32cb29e` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:25.450293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc54ee5f6c234338ae7344f8d32cb29e)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:25.452472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:25.492413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:25.677295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc54ee5f6c234338ae7344f8d32cb29e', 'request_id': 'req-19cfd0c0-0f7a-4d59-8bda-6793bd316752'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:25.791872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc54ee5f6c234338ae7344f8d32cb29e', 'request_id': 'req-19cfd0c0-0f7a-4d59-8bda-6793bd316752'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:25.818542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc54ee5f6c234338ae7344f8d32cb29e', 'request_id': 'req-19cfd0c0-0f7a-4d59-8bda-6793bd316752'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:25.849263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] The token cache is being invalidate because project fc54ee5f6c234338ae7344f8d32cb29e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:25.849918 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:25.851111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-19cfd0c0-0f7a-4d59-8bda-6793bd316752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:25.862631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8b99e-e296-4f9d-891f-02e194819f29 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:25.863443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8b99e-e296-4f9d-891f-02e194819f29 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:25.863443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8b99e-e296-4f9d-891f-02e194819f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:25.867493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 995/1992] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:25 2022] DELETE /identity/v3/projects/fc54ee5f6c234338ae7344f8d32cb29e => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:25.893077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-fed8b99e-e296-4f9d-891f-02e194819f29 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:25.894736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fed8b99e-e296-4f9d-891f-02e194819f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:25.951673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:25.991737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 998/1993] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:25.997462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab33e82-5a64-43f3-9d46-d4dfbd43681b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:25.997872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab33e82-5a64-43f3-9d46-d4dfbd43681b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:25.998332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab33e82-5a64-43f3-9d46-d4dfbd43681b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:26.035587 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-4ab33e82-5a64-43f3-9d46-d4dfbd43681b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:26.038258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4ab33e82-5a64-43f3-9d46-d4dfbd43681b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:26.051312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 996/1994] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:26.055340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8b99e-e296-4f9d-891f-02e194819f29 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:26.066733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8b99e-e296-4f9d-891f-02e194819f29 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:26.073199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c9fe0-3ee3-45db-a43b-0d5205ce0b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:26.074506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c9fe0-3ee3-45db-a43b-0d5205ce0b28 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:26.074825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c9fe0-3ee3-45db-a43b-0d5205ce0b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:26.075094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c9fe0-3ee3-45db-a43b-0d5205ce0b28 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:26.075764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c9fe0-3ee3-45db-a43b-0d5205ce0b28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:26.076065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c9fe0-3ee3-45db-a43b-0d5205ce0b28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:26.077909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c9fe0-3ee3-45db-a43b-0d5205ce0b28 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:26.091187 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 999/1995] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:53:26.095911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab33e82-5a64-43f3-9d46-d4dfbd43681b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:26.106573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab33e82-5a64-43f3-9d46-d4dfbd43681b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:26.111857 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a505e7-9774-46e2-a2f1-366b56053cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:26.112474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a505e7-9774-46e2-a2f1-366b56053cde admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:26.112752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a505e7-9774-46e2-a2f1-366b56053cde admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:26.113549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a505e7-9774-46e2-a2f1-366b56053cde admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:26.113691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a505e7-9774-46e2-a2f1-366b56053cde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:26.115834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a505e7-9774-46e2-a2f1-366b56053cde admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:26.227799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c9fe0-3ee3-45db-a43b-0d5205ce0b28 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:26.237536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 997/1996] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:26 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:26.242497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c9fe0-3ee3-45db-a43b-0d5205ce0b28 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:26.322354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca89c0a-1f27-4361-b378-5593a075b8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:26.323240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca89c0a-1f27-4361-b378-5593a075b8fa admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:26.323531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca89c0a-1f27-4361-b378-5593a075b8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:26.324391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca89c0a-1f27-4361-b378-5593a075b8fa admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:26.325359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca89c0a-1f27-4361-b378-5593a075b8fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:26.327895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca89c0a-1f27-4361-b378-5593a075b8fa admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:26.421868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1000/1997] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:26 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:26.426064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a505e7-9774-46e2-a2f1-366b56053cde admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:26.436904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a505e7-9774-46e2-a2f1-366b56053cde admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:26.441798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504e717-e974-41d4-a7eb-72f85cc8f80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:26.442446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504e717-e974-41d4-a7eb-72f85cc8f80d admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:26.442722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504e717-e974-41d4-a7eb-72f85cc8f80d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:26.445992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9504e717-e974-41d4-a7eb-72f85cc8f80d admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:26.445992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504e717-e974-41d4-a7eb-72f85cc8f80d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:26.445992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9504e717-e974-41d4-a7eb-72f85cc8f80d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:26.461918 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 998/1998] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:26.462481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca89c0a-1f27-4361-b378-5593a075b8fa admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:26.475855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca89c0a-1f27-4361-b378-5593a075b8fa admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:26.481829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab0265-73ab-4683-922f-a4b1b18c4610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:26.482487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab0265-73ab-4683-922f-a4b1b18c4610 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:26.482823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab0265-73ab-4683-922f-a4b1b18c4610 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:26.483094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab0265-73ab-4683-922f-a4b1b18c4610 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:26.483746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab0265-73ab-4683-922f-a4b1b18c4610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:26.485413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab0265-73ab-4683-922f-a4b1b18c4610 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:26.497385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504e717-e974-41d4-a7eb-72f85cc8f80d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:26.499512 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9504e717-e974-41d4-a7eb-72f85cc8f80d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:26.502512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1001/1999] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:26.508688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab0265-73ab-4683-922f-a4b1b18c4610 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:26.515337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9504e717-e974-41d4-a7eb-72f85cc8f80d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:26.520644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:26.521317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:26.521604 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:26.521873 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] PATH_INFO: `/v3/users/3903ea3c41734facbb54bf671f03ec97` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:26.523268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3903ea3c41734facbb54bf671f03ec97)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:26.525543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:26.532614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab0265-73ab-4683-922f-a4b1b18c4610 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:26.544632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6360ed8-3b3e-462e-a01e-80fc898dac6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:26.545313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6360ed8-3b3e-462e-a01e-80fc898dac6d admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:26.545591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6360ed8-3b3e-462e-a01e-80fc898dac6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:26.545862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6360ed8-3b3e-462e-a01e-80fc898dac6d admin admin] PATH_INFO: `/v3/projects/7b75fb609a4d4df3b23e88d85a79650a/users/7d77a70ecf2c4657a09caeda776743c4/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:26.570567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6360ed8-3b3e-462e-a01e-80fc898dac6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b75fb609a4d4df3b23e88d85a79650a, user_id=7d77a70ecf2c4657a09caeda776743c4, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:26.573918 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6360ed8-3b3e-462e-a01e-80fc898dac6d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:26.712043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1002/2000] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:26 2022] PUT /identity/v3/projects/7b75fb609a4d4df3b23e88d85a79650a/users/7d77a70ecf2c4657a09caeda776743c4/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:26.717263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714b723-3525-414b-bcee-0a5e192043c6 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:26.717556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714b723-3525-414b-bcee-0a5e192043c6 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:26.717833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714b723-3525-414b-bcee-0a5e192043c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:26.747720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c714b723-3525-414b-bcee-0a5e192043c6 None None] MFA Rules not processed for user `7d77a70ecf2c4657a09caeda776743c4`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:26.751734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c714b723-3525-414b-bcee-0a5e192043c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:26.803104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3903ea3c41734facbb54bf671f03ec97'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:26.846105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1003/2001] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:26.859229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714b723-3525-414b-bcee-0a5e192043c6 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:26.862448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c714b723-3525-414b-bcee-0a5e192043c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:26.863031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3903ea3c41734facbb54bf671f03ec97'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:26.873933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3903ea3c41734facbb54bf671f03ec97'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:26.910369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3903ea3c41734facbb54bf671f03ec97'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:26.914750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-be4f7dd8-259f-40e5-9cc8-d5b2e7dde19a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3903ea3c41734facbb54bf671f03ec97'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:26.930044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 999/2002] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:26 2022] DELETE /identity/v3/users/3903ea3c41734facbb54bf671f03ec97 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:26.934835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a870-a4bb-45a0-a25e-a5fc2efba59e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:26.935315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a870-a4bb-45a0-a25e-a5fc2efba59e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:26.935764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a870-a4bb-45a0-a25e-a5fc2efba59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:26.981460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ce77a870-a4bb-45a0-a25e-a5fc2efba59e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:26.984440 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ce77a870-a4bb-45a0-a25e-a5fc2efba59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:27.035975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714b723-3525-414b-bcee-0a5e192043c6 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:27.125481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1000/2003] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:27.137663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77a870-a4bb-45a0-a25e-a5fc2efba59e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:27.140241 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:27.141222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ce77a870-a4bb-45a0-a25e-a5fc2efba59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:27.190428 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107b3b2-1364-4507-84e9-9eda05431656 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:27.191052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107b3b2-1364-4507-84e9-9eda05431656 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:27.191326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107b3b2-1364-4507-84e9-9eda05431656 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:27.191590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107b3b2-1364-4507-84e9-9eda05431656 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:27.204867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4107b3b2-1364-4507-84e9-9eda05431656 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:27.294356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:27.342701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77a870-a4bb-45a0-a25e-a5fc2efba59e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:27.354722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce82d92-41d6-4b01-95b6-6d4d95027f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:27.355943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce82d92-41d6-4b01-95b6-6d4d95027f17 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:27.356224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce82d92-41d6-4b01-95b6-6d4d95027f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:27.356494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce82d92-41d6-4b01-95b6-6d4d95027f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:27.373092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107b3b2-1364-4507-84e9-9eda05431656 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:27.375803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107b3b2-1364-4507-84e9-9eda05431656 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:27.382826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce82d92-41d6-4b01-95b6-6d4d95027f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:27.384301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce82d92-41d6-4b01-95b6-6d4d95027f17 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:27.404451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1004/2004] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:53:27.408622 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1001/2005] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:53:27.492889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7b263-97c8-43e8-843f-048d1117f80b None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:27.493176 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7b263-97c8-43e8-843f-048d1117f80b None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:27.493450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7b263-97c8-43e8-843f-048d1117f80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:27.556398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-bdf7b263-97c8-43e8-843f-048d1117f80b None None] MFA Rules not processed for user `8d77f31d797448b6ad4964fa96e2d492`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:27.560183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bdf7b263-97c8-43e8-843f-048d1117f80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:27.684703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1005/2006] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:27.690051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce82d92-41d6-4b01-95b6-6d4d95027f17 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:27.691965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5ce82d92-41d6-4b01-95b6-6d4d95027f17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:27.704527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce82d92-41d6-4b01-95b6-6d4d95027f17 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:27.709068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8456d7-c1d8-4f54-8307-a4e0bd41ebfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:27.709588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8456d7-c1d8-4f54-8307-a4e0bd41ebfe admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:27.709859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8456d7-c1d8-4f54-8307-a4e0bd41ebfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:27.710178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8456d7-c1d8-4f54-8307-a4e0bd41ebfe admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:27.710795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8456d7-c1d8-4f54-8307-a4e0bd41ebfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:27.712654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8456d7-c1d8-4f54-8307-a4e0bd41ebfe admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:27.870558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1002/2007] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:27 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:27.870721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7b263-97c8-43e8-843f-048d1117f80b None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:27.876966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7b263-97c8-43e8-843f-048d1117f80b None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:27.936294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e20bd-cba1-4073-98b6-0f3bc520570d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:27.936893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e20bd-cba1-4073-98b6-0f3bc520570d admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:27.937167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e20bd-cba1-4073-98b6-0f3bc520570d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:27.937432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e20bd-cba1-4073-98b6-0f3bc520570d admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:27.938105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e20bd-cba1-4073-98b6-0f3bc520570d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:27.940005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e20bd-cba1-4073-98b6-0f3bc520570d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:28.061312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1006/2008] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:27 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:28.065501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e20bd-cba1-4073-98b6-0f3bc520570d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:28.065658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8456d7-c1d8-4f54-8307-a4e0bd41ebfe admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:28.065658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9e8456d7-c1d8-4f54-8307-a4e0bd41ebfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:28.074379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e20bd-cba1-4073-98b6-0f3bc520570d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:28.079841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dedaec-bf5b-4643-95ca-0f4c85d6915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:28.080636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dedaec-bf5b-4643-95ca-0f4c85d6915c admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.080966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dedaec-bf5b-4643-95ca-0f4c85d6915c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.081625 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dedaec-bf5b-4643-95ca-0f4c85d6915c admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.082441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dedaec-bf5b-4643-95ca-0f4c85d6915c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:28.088345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dedaec-bf5b-4643-95ca-0f4c85d6915c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:28.092595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8456d7-c1d8-4f54-8307-a4e0bd41ebfe admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:28.106433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:28.107075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.107354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.107680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] PATH_INFO: `/v3/projects/004336e4f4f347918a221ef96db56a93` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.108811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=004336e4f4f347918a221ef96db56a93)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:28.110779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:28.112983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1007/2009] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:28.117538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dedaec-bf5b-4643-95ca-0f4c85d6915c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:28.135106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dedaec-bf5b-4643-95ca-0f4c85d6915c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:28.140439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5074dd-f02a-45e3-8bea-ad6efcc774ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:28.141143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5074dd-f02a-45e3-8bea-ad6efcc774ac admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.141421 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5074dd-f02a-45e3-8bea-ad6efcc774ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.141809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5074dd-f02a-45e3-8bea-ad6efcc774ac admin admin] PATH_INFO: `/v3/projects/c0d7096c1d1b47ffbaed0d337c04a190/users/59cd44da419947bb8df30e59f0faf50b/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.159775 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5074dd-f02a-45e3-8bea-ad6efcc774ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0d7096c1d1b47ffbaed0d337c04a190, user_id=59cd44da419947bb8df30e59f0faf50b, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:28.162483 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5074dd-f02a-45e3-8bea-ad6efcc774ac admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:28.259534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1008/2010] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:28 2022] PUT /identity/v3/projects/c0d7096c1d1b47ffbaed0d337c04a190/users/59cd44da419947bb8df30e59f0faf50b/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:28.264803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5074dd-f02a-45e3-8bea-ad6efcc774ac admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:28.278081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004336e4f4f347918a221ef96db56a93', 'request_id': 'req-965730e7-adac-4725-8f5c-f8ffc6c89de9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:28.333437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5074dd-f02a-45e3-8bea-ad6efcc774ac admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:28.408802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78bce6-5bc6-4372-9f0b-c09df7332608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:28.409485 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78bce6-5bc6-4372-9f0b-c09df7332608 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.409776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78bce6-5bc6-4372-9f0b-c09df7332608 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.410094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78bce6-5bc6-4372-9f0b-c09df7332608 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.410889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78bce6-5bc6-4372-9f0b-c09df7332608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:28.411295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004336e4f4f347918a221ef96db56a93', 'request_id': 'req-965730e7-adac-4725-8f5c-f8ffc6c89de9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:28.412667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78bce6-5bc6-4372-9f0b-c09df7332608 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:28.419164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004336e4f4f347918a221ef96db56a93', 'request_id': 'req-965730e7-adac-4725-8f5c-f8ffc6c89de9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:28.425084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1009/2011] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:28.428038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78bce6-5bc6-4372-9f0b-c09df7332608 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:28.431984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] The token cache is being invalidate because project 004336e4f4f347918a221ef96db56a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:28.432476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:28.433836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-965730e7-adac-4725-8f5c-f8ffc6c89de9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:28.435519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1003/2012] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:28 2022] DELETE /identity/v3/projects/004336e4f4f347918a221ef96db56a93 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:28.440931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a7ff7-8247-4dcd-a1fd-a4725d02604e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.441284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a7ff7-8247-4dcd-a1fd-a4725d02604e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.441476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a7ff7-8247-4dcd-a1fd-a4725d02604e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.447293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78bce6-5bc6-4372-9f0b-c09df7332608 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:28.452019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:28.455359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394611e-215f-4cd9-bb1e-42a160f068e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:28.455991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394611e-215f-4cd9-bb1e-42a160f068e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.456269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394611e-215f-4cd9-bb1e-42a160f068e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.456538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394611e-215f-4cd9-bb1e-42a160f068e1 admin admin] PATH_INFO: `/v3/projects/c0d7096c1d1b47ffbaed0d337c04a190/users/59cd44da419947bb8df30e59f0faf50b/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.460054 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394611e-215f-4cd9-bb1e-42a160f068e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0d7096c1d1b47ffbaed0d337c04a190, user_id=59cd44da419947bb8df30e59f0faf50b, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:28.461791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394611e-215f-4cd9-bb1e-42a160f068e1 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:28.468615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-246a7ff7-8247-4dcd-a1fd-a4725d02604e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:28.469856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-246a7ff7-8247-4dcd-a1fd-a4725d02604e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:28.482977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1010/2013] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:28 2022] PUT /identity/v3/projects/c0d7096c1d1b47ffbaed0d337c04a190/users/59cd44da419947bb8df30e59f0faf50b/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:28.547404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1004/2014] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:28.555871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909f98b-fd26-486f-b0d0-be454bdde9b5 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.555871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909f98b-fd26-486f-b0d0-be454bdde9b5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.556090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909f98b-fd26-486f-b0d0-be454bdde9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.580856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d909f98b-fd26-486f-b0d0-be454bdde9b5 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:28.582208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d909f98b-fd26-486f-b0d0-be454bdde9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:28.591006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1011/2015] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:28.595626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a7ff7-8247-4dcd-a1fd-a4725d02604e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:28.605896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a7ff7-8247-4dcd-a1fd-a4725d02604e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:28.611537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0b9a7-0efc-4d09-a621-11fce3d9e026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:28.612225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0b9a7-0efc-4d09-a621-11fce3d9e026 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.612574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0b9a7-0efc-4d09-a621-11fce3d9e026 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.612910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0b9a7-0efc-4d09-a621-11fce3d9e026 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.613671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0b9a7-0efc-4d09-a621-11fce3d9e026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:28.614065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0b9a7-0efc-4d09-a621-11fce3d9e026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:28.615966 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0b9a7-0efc-4d09-a621-11fce3d9e026 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:28.629985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1005/2016] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:53:28.633943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909f98b-fd26-486f-b0d0-be454bdde9b5 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:28.643533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d909f98b-fd26-486f-b0d0-be454bdde9b5 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:28.648236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4ee19-2f76-48f1-8237-2432a99d58a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:28.648886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4ee19-2f76-48f1-8237-2432a99d58a9 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.649166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4ee19-2f76-48f1-8237-2432a99d58a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.649442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4ee19-2f76-48f1-8237-2432a99d58a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.650126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4ee19-2f76-48f1-8237-2432a99d58a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:28.652292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4ee19-2f76-48f1-8237-2432a99d58a9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:28.754874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:28.805701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1012/2017] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:28 2022] POST /identity/v3/projects => generated 397 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:28.806618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0b9a7-0efc-4d09-a621-11fce3d9e026 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:28.817543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0b9a7-0efc-4d09-a621-11fce3d9e026 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:28.884375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e247229-7f5d-4e43-ba6a-211ed7d75b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:28.885712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e247229-7f5d-4e43-ba6a-211ed7d75b06 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.885712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e247229-7f5d-4e43-ba6a-211ed7d75b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.885928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e247229-7f5d-4e43-ba6a-211ed7d75b06 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.887028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e247229-7f5d-4e43-ba6a-211ed7d75b06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:28.889159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e247229-7f5d-4e43-ba6a-211ed7d75b06 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:28.968049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1006/2018] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:28 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:28.972481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4ee19-2f76-48f1-8237-2432a99d58a9 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:28.984915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4ee19-2f76-48f1-8237-2432a99d58a9 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:28.989857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6102f-e434-4d37-acb7-de1db0b21fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:28.990467 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6102f-e434-4d37-acb7-de1db0b21fba admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:28.990719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6102f-e434-4d37-acb7-de1db0b21fba admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:28.990989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6102f-e434-4d37-acb7-de1db0b21fba admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:28.991638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6102f-e434-4d37-acb7-de1db0b21fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:28.993246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6102f-e434-4d37-acb7-de1db0b21fba admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:29.004561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1013/2019] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:29.008840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e247229-7f5d-4e43-ba6a-211ed7d75b06 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:29.020553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e247229-7f5d-4e43-ba6a-211ed7d75b06 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:29.035662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80b651-6ded-4795-ad97-1c8e2bbf56bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:29.035662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80b651-6ded-4795-ad97-1c8e2bbf56bd admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:29.035662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80b651-6ded-4795-ad97-1c8e2bbf56bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:29.035662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80b651-6ded-4795-ad97-1c8e2bbf56bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:29.036675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80b651-6ded-4795-ad97-1c8e2bbf56bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:29.037832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80b651-6ded-4795-ad97-1c8e2bbf56bd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:29.051494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1007/2020] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:29.056403 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6102f-e434-4d37-acb7-de1db0b21fba admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:29.068900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6102f-e434-4d37-acb7-de1db0b21fba admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:29.075373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32580dae-64cb-474b-a38a-44dfba13b33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:29.076035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32580dae-64cb-474b-a38a-44dfba13b33d admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:29.076360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32580dae-64cb-474b-a38a-44dfba13b33d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:29.076634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32580dae-64cb-474b-a38a-44dfba13b33d admin admin] PATH_INFO: `/v3/projects/54c43bdef3b24686b28b10b1581becae/users/962e349dd4c1469db54adbd7433025df/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:29.093130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32580dae-64cb-474b-a38a-44dfba13b33d admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c43bdef3b24686b28b10b1581becae, user_id=962e349dd4c1469db54adbd7433025df, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:29.095598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32580dae-64cb-474b-a38a-44dfba13b33d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:29.155369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:29.189880 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1014/2021] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:29 2022] PUT /identity/v3/projects/54c43bdef3b24686b28b10b1581becae/users/962e349dd4c1469db54adbd7433025df/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:29.204093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89d83c-a399-4533-869c-d9b87c4001e4 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:29.204556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89d83c-a399-4533-869c-d9b87c4001e4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:29.204828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89d83c-a399-4533-869c-d9b87c4001e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:29.243998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5f89d83c-a399-4533-869c-d9b87c4001e4 None None] MFA Rules not processed for user `962e349dd4c1469db54adbd7433025df`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:29.246214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5f89d83c-a399-4533-869c-d9b87c4001e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:29.338347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1008/2022] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:29 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:29.348216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32580dae-64cb-474b-a38a-44dfba13b33d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:29.350105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-32580dae-64cb-474b-a38a-44dfba13b33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:29.422387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32580dae-64cb-474b-a38a-44dfba13b33d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:29.426925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44123a39-ee5f-43d1-967f-50b416305fb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:29.427683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44123a39-ee5f-43d1-967f-50b416305fb7 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:29.427841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44123a39-ee5f-43d1-967f-50b416305fb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:29.428144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44123a39-ee5f-43d1-967f-50b416305fb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:29.446245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44123a39-ee5f-43d1-967f-50b416305fb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:29.447854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44123a39-ee5f-43d1-967f-50b416305fb7 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:29.458685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1015/2023] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:29 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:34.664178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516d330-9c1d-4c7c-89e2-453723da5d6a None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:34.666564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516d330-9c1d-4c7c-89e2-453723da5d6a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:34.666564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516d330-9c1d-4c7c-89e2-453723da5d6a None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:34.666564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1009/2024] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 16:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:53:34.687778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472c02-9221-4848-bc5c-e815853f28d7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:34.688418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472c02-9221-4848-bc5c-e815853f28d7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:34.688418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472c02-9221-4848-bc5c-e815853f28d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:34.722398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-1c472c02-9221-4848-bc5c-e815853f28d7 None None] MFA Rules not processed for user `d7bc938254514471801c88761525e883`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:34.724302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-1c472c02-9221-4848-bc5c-e815853f28d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:34.803450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1016/2025] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 16:53:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:34.828350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516d330-9c1d-4c7c-89e2-453723da5d6a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:34.830305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2516d330-9c1d-4c7c-89e2-453723da5d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:34.866222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516d330-9c1d-4c7c-89e2-453723da5d6a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:34.884722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ff550-f94a-4327-9433-e5e338737f2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:34.885353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ff550-f94a-4327-9433-e5e338737f2c service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:34.885660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ff550-f94a-4327-9433-e5e338737f2c service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:34.885922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ff550-f94a-4327-9433-e5e338737f2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:34.908809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c472c02-9221-4848-bc5c-e815853f28d7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:34.910497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-1c472c02-9221-4848-bc5c-e815853f28d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:34.910835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ff550-f94a-4327-9433-e5e338737f2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:34.914955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ff550-f94a-4327-9433-e5e338737f2c service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:34.926602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1010/2026] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:34.991886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:35.018501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c472c02-9221-4848-bc5c-e815853f28d7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:35.038174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:35.038948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:35.039203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:35.039482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] PATH_INFO: `/v3/users/f4ee83b28c5f417792da4ab321a174d0` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:35.040683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ee83b28c5f417792da4ab321a174d0)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:35.043216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:35.296345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ee83b28c5f417792da4ab321a174d0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:35.385133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ee83b28c5f417792da4ab321a174d0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:35.400139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ee83b28c5f417792da4ab321a174d0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:35.403006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ee83b28c5f417792da4ab321a174d0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:35.403790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ee83b28c5f417792da4ab321a174d0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:35.425561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1017/2027] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:34 2022] DELETE /identity/v3/users/f4ee83b28c5f417792da4ab321a174d0 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:35.755079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb77f9f-b479-4031-8249-da014ea39c0b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:35.755573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb77f9f-b479-4031-8249-da014ea39c0b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:35.755653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb77f9f-b479-4031-8249-da014ea39c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:35.787903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ceb77f9f-b479-4031-8249-da014ea39c0b None None] MFA Rules not processed for user `59cd44da419947bb8df30e59f0faf50b`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:35.789701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ceb77f9f-b479-4031-8249-da014ea39c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:35.807556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:35.808067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:35.810030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:35.885240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1011/2028] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:35.895548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1349ca-e1bc-48b6-b30f-cb00f8b3a8f9 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:35.899682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb77f9f-b479-4031-8249-da014ea39c0b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:35.902267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ceb77f9f-b479-4031-8249-da014ea39c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:35.903072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:35.903711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:35.903984 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:35.904248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] PATH_INFO: `/v3/projects/c5aead202fae47d791fe9a58b8c8faf1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:35.905358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5aead202fae47d791fe9a58b8c8faf1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:35.907281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:36.012439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb77f9f-b479-4031-8249-da014ea39c0b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:36.017506 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa532719-e0d4-4a44-87d7-e15dc07f7825 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33003a9bf4454f5bac2e6ad73544efc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:36.018400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa532719-e0d4-4a44-87d7-e15dc07f7825 service glance] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.018853 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa532719-e0d4-4a44-87d7-e15dc07f7825 service glance] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.019270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa532719-e0d4-4a44-87d7-e15dc07f7825 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.024695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-aa532719-e0d4-4a44-87d7-e15dc07f7825 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:36.043462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5aead202fae47d791fe9a58b8c8faf1', 'request_id': 'req-9c750cd0-a404-4090-90d8-0d4aa6487420'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:36.092840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa532719-e0d4-4a44-87d7-e15dc07f7825 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:36.094381 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa532719-e0d4-4a44-87d7-e15dc07f7825 service glance] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:36.104676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1012/2029] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:36.111960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5aead202fae47d791fe9a58b8c8faf1', 'request_id': 'req-9c750cd0-a404-4090-90d8-0d4aa6487420'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:36.119200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5aead202fae47d791fe9a58b8c8faf1', 'request_id': 'req-9c750cd0-a404-4090-90d8-0d4aa6487420'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:36.120991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:36.130671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] The token cache is being invalidate because project c5aead202fae47d791fe9a58b8c8faf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:36.130963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:36.131650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9c750cd0-a404-4090-90d8-0d4aa6487420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:36.133176 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1018/2030] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:35 2022] DELETE /identity/v3/projects/c5aead202fae47d791fe9a58b8c8faf1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:36.138697 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8579a2-3270-4646-80fa-8d6553501bb6 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.139137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8579a2-3270-4646-80fa-8d6553501bb6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.139418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8579a2-3270-4646-80fa-8d6553501bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.176391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-7f8579a2-3270-4646-80fa-8d6553501bb6 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:36.176391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7f8579a2-3270-4646-80fa-8d6553501bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:36.218034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:36.251846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1013/2031] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:36.259989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c130bf3-ff2d-45aa-9be2-bbb4a2583715 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.260276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c130bf3-ff2d-45aa-9be2-bbb4a2583715 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.261150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c130bf3-ff2d-45aa-9be2-bbb4a2583715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.301393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9c130bf3-ff2d-45aa-9be2-bbb4a2583715 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:36.303539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9c130bf3-ff2d-45aa-9be2-bbb4a2583715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:36.316462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1019/2032] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:36.320478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8579a2-3270-4646-80fa-8d6553501bb6 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:36.333875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8579a2-3270-4646-80fa-8d6553501bb6 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:36.339228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dff6a-b428-4195-9731-c1762f81a68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:36.339959 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dff6a-b428-4195-9731-c1762f81a68c admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.340341 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dff6a-b428-4195-9731-c1762f81a68c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.340711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dff6a-b428-4195-9731-c1762f81a68c admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.341651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dff6a-b428-4195-9731-c1762f81a68c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:36.342089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dff6a-b428-4195-9731-c1762f81a68c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:36.343807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dff6a-b428-4195-9731-c1762f81a68c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:36.362363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1014/2033] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:53:36.367073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c130bf3-ff2d-45aa-9be2-bbb4a2583715 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:36.378587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c130bf3-ff2d-45aa-9be2-bbb4a2583715 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:36.383390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d8c22-2016-4723-aec9-88274ca29a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:36.384041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d8c22-2016-4723-aec9-88274ca29a0b admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.384323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d8c22-2016-4723-aec9-88274ca29a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.384594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d8c22-2016-4723-aec9-88274ca29a0b admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.385253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d8c22-2016-4723-aec9-88274ca29a0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:36.387938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d8c22-2016-4723-aec9-88274ca29a0b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:36.464925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91223f-4b56-45bc-b3e3-94c3b9e9973a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.465239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91223f-4b56-45bc-b3e3-94c3b9e9973a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.465599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91223f-4b56-45bc-b3e3-94c3b9e9973a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.485417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9e91223f-4b56-45bc-b3e3-94c3b9e9973a None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:36.488625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9e91223f-4b56-45bc-b3e3-94c3b9e9973a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:36.518217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1020/2034] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:36 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:36.521224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d8c22-2016-4723-aec9-88274ca29a0b admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:36.549378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1015/2035] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 16:53:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:36.594188 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91223f-4b56-45bc-b3e3-94c3b9e9973a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:36.594929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9e91223f-4b56-45bc-b3e3-94c3b9e9973a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:36.595292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d8c22-2016-4723-aec9-88274ca29a0b admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:36.599892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7c9df-94b5-4bd8-9ecd-c20ae42181d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:36.600665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7c9df-94b5-4bd8-9ecd-c20ae42181d2 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.600940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7c9df-94b5-4bd8-9ecd-c20ae42181d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.601209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7c9df-94b5-4bd8-9ecd-c20ae42181d2 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.601853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7c9df-94b5-4bd8-9ecd-c20ae42181d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:36.603931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de7c9df-94b5-4bd8-9ecd-c20ae42181d2 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:36.671041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e91223f-4b56-45bc-b3e3-94c3b9e9973a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:36.708469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1021/2036] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:36 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:36.713080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7c9df-94b5-4bd8-9ecd-c20ae42181d2 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:36.723943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7c9df-94b5-4bd8-9ecd-c20ae42181d2 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:36.728809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36570d-4f26-45df-8ce9-628b15ad9c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:36.729940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36570d-4f26-45df-8ce9-628b15ad9c6b admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.730241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36570d-4f26-45df-8ce9-628b15ad9c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.730509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d36570d-4f26-45df-8ce9-628b15ad9c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.731155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36570d-4f26-45df-8ce9-628b15ad9c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:36.732773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d36570d-4f26-45df-8ce9-628b15ad9c6b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:36.736268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d60fd-f90e-4123-a777-77e2e7fef945 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:36.736874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d60fd-f90e-4123-a777-77e2e7fef945 service swift] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.737168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d60fd-f90e-4123-a777-77e2e7fef945 service swift] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.737457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d60fd-f90e-4123-a777-77e2e7fef945 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.743001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1022/2037] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:36.747956 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36570d-4f26-45df-8ce9-628b15ad9c6b admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:36.757894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d36570d-4f26-45df-8ce9-628b15ad9c6b admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:36.758194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:36.763205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389ac1c-c51b-4270-8c5f-1de1e2d0a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:36.763797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389ac1c-c51b-4270-8c5f-1de1e2d0a998 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.764089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389ac1c-c51b-4270-8c5f-1de1e2d0a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.764341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389ac1c-c51b-4270-8c5f-1de1e2d0a998 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.765000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389ac1c-c51b-4270-8c5f-1de1e2d0a998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:36.766654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389ac1c-c51b-4270-8c5f-1de1e2d0a998 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:36.799692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1023/2038] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:36.800053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d60fd-f90e-4123-a777-77e2e7fef945 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:36.801325 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d60fd-f90e-4123-a777-77e2e7fef945 service swift] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:36.803137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389ac1c-c51b-4270-8c5f-1de1e2d0a998 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:36.809945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1016/2039] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 16:53:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 09 16:53:36.814262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389ac1c-c51b-4270-8c5f-1de1e2d0a998 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:36.819701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec4e53-25cf-40b0-b7d8-dd1ce6156d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:36.820395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec4e53-25cf-40b0-b7d8-dd1ce6156d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.820761 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec4e53-25cf-40b0-b7d8-dd1ce6156d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.821038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec4e53-25cf-40b0-b7d8-dd1ce6156d29 admin admin] PATH_INFO: `/v3/projects/1137426f1915403fa7b008d6e9b7e20a/users/2ebfde4a905d42389abb42bf33ba426d/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.844488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec4e53-25cf-40b0-b7d8-dd1ce6156d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1137426f1915403fa7b008d6e9b7e20a, user_id=2ebfde4a905d42389abb42bf33ba426d, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:36.847193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec4e53-25cf-40b0-b7d8-dd1ce6156d29 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:36.921934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1024/2040] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:36 2022] PUT /identity/v3/projects/1137426f1915403fa7b008d6e9b7e20a/users/2ebfde4a905d42389abb42bf33ba426d/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:36.932619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fd225-2ca2-4774-a29a-ad5f07a86ab6 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:36.933200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fd225-2ca2-4774-a29a-ad5f07a86ab6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:36.933609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fd225-2ca2-4774-a29a-ad5f07a86ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:36.972416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a17fd225-2ca2-4774-a29a-ad5f07a86ab6 None None] MFA Rules not processed for user `2ebfde4a905d42389abb42bf33ba426d`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:36.973805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a17fd225-2ca2-4774-a29a-ad5f07a86ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:37.070918 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1017/2041] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:37.074858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec4e53-25cf-40b0-b7d8-dd1ce6156d29 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:37.174207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec4e53-25cf-40b0-b7d8-dd1ce6156d29 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:37.186430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154529a3-6831-45c1-b55f-bb3bc871a59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:37.186430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154529a3-6831-45c1-b55f-bb3bc871a59a admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:37.186430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154529a3-6831-45c1-b55f-bb3bc871a59a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:37.186430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154529a3-6831-45c1-b55f-bb3bc871a59a admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:37.187636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154529a3-6831-45c1-b55f-bb3bc871a59a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:37.189476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154529a3-6831-45c1-b55f-bb3bc871a59a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:37.206501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:37.671036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1025/2042] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:37 2022] POST /identity/v3/projects => generated 387 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:37.671281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fd225-2ca2-4774-a29a-ad5f07a86ab6 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:37.687282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fd225-2ca2-4774-a29a-ad5f07a86ab6 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:37.773480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4a7a1-b138-4383-aac9-cc10fd09a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:37.774306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4a7a1-b138-4383-aac9-cc10fd09a90b admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:37.774395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4a7a1-b138-4383-aac9-cc10fd09a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:37.776769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d4a7a1-b138-4383-aac9-cc10fd09a90b admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:37.776769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4a7a1-b138-4383-aac9-cc10fd09a90b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:37.777558 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d4a7a1-b138-4383-aac9-cc10fd09a90b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:37.989063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1018/2043] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:37 2022] POST /identity/v3/users => generated 300 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:37.989596 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154529a3-6831-45c1-b55f-bb3bc871a59a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:37.993340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154529a3-6831-45c1-b55f-bb3bc871a59a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:38.005360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807c70b-3905-45c5-88d8-1fcc51148557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:38.006006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807c70b-3905-45c5-88d8-1fcc51148557 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:38.006319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807c70b-3905-45c5-88d8-1fcc51148557 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:38.006588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807c70b-3905-45c5-88d8-1fcc51148557 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:38.007226 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807c70b-3905-45c5-88d8-1fcc51148557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:38.008902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807c70b-3905-45c5-88d8-1fcc51148557 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:38.025482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1026/2044] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:38.029903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4a7a1-b138-4383-aac9-cc10fd09a90b admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:38.047186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d4a7a1-b138-4383-aac9-cc10fd09a90b admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:38.052536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288e131-d2cf-44c0-8aa6-20123289dd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:38.053342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288e131-d2cf-44c0-8aa6-20123289dd2b admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:38.054221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288e131-d2cf-44c0-8aa6-20123289dd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:38.054610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288e131-d2cf-44c0-8aa6-20123289dd2b admin admin] PATH_INFO: `/v3/projects/a85c30a1cbf74311a0c42927ced737a5/users/88a2ec6d55914aa4924a4d5c7f104a1e/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:38.081001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288e131-d2cf-44c0-8aa6-20123289dd2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85c30a1cbf74311a0c42927ced737a5, user_id=88a2ec6d55914aa4924a4d5c7f104a1e, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:38.083751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288e131-d2cf-44c0-8aa6-20123289dd2b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:38.199212 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1019/2045] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:38 2022] PUT /identity/v3/projects/a85c30a1cbf74311a0c42927ced737a5/users/88a2ec6d55914aa4924a4d5c7f104a1e/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:38.203012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807c70b-3905-45c5-88d8-1fcc51148557 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:38.223672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807c70b-3905-45c5-88d8-1fcc51148557 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:38.301728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1315d-c2a0-44e7-8821-40e6ad2e25da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:38.302583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1315d-c2a0-44e7-8821-40e6ad2e25da admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:38.302961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1315d-c2a0-44e7-8821-40e6ad2e25da admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:38.303336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1315d-c2a0-44e7-8821-40e6ad2e25da admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:38.304188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1315d-c2a0-44e7-8821-40e6ad2e25da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:38.306066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1315d-c2a0-44e7-8821-40e6ad2e25da admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:38.323636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288e131-d2cf-44c0-8aa6-20123289dd2b admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:38.325805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1027/2046] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:38.341379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288e131-d2cf-44c0-8aa6-20123289dd2b admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:38.346873 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c1515-ab80-4a21-b16a-ff9697e57b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:38.347641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c1515-ab80-4a21-b16a-ff9697e57b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:38.348020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c1515-ab80-4a21-b16a-ff9697e57b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:38.348517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c1515-ab80-4a21-b16a-ff9697e57b0f admin admin] PATH_INFO: `/v3/projects/a85c30a1cbf74311a0c42927ced737a5/users/88a2ec6d55914aa4924a4d5c7f104a1e/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:38.350656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c1515-ab80-4a21-b16a-ff9697e57b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85c30a1cbf74311a0c42927ced737a5, user_id=88a2ec6d55914aa4924a4d5c7f104a1e, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:38.353479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c1515-ab80-4a21-b16a-ff9697e57b0f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:38.364452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:38.379256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1020/2047] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:38 2022] PUT /identity/v3/projects/a85c30a1cbf74311a0c42927ced737a5/users/88a2ec6d55914aa4924a4d5c7f104a1e/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:38.383290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa357eab-a0c9-4ab3-9e33-166ff7ac4d46 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:38.383676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa357eab-a0c9-4ab3-9e33-166ff7ac4d46 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:38.385053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa357eab-a0c9-4ab3-9e33-166ff7ac4d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:38.419449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-fa357eab-a0c9-4ab3-9e33-166ff7ac4d46 None None] MFA Rules not processed for user `88a2ec6d55914aa4924a4d5c7f104a1e`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:38.420809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fa357eab-a0c9-4ab3-9e33-166ff7ac4d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:38.509592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1028/2048] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:38.533732 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c1515-ab80-4a21-b16a-ff9697e57b0f admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:38.537502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-735c1515-ab80-4a21-b16a-ff9697e57b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:38.620645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c1515-ab80-4a21-b16a-ff9697e57b0f admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:38.627808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a52084-e4e7-4bf0-b27f-49ad3821297c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:38.628435 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a52084-e4e7-4bf0-b27f-49ad3821297c service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:38.628856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a52084-e4e7-4bf0-b27f-49ad3821297c service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:38.629140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a52084-e4e7-4bf0-b27f-49ad3821297c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:38.642248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a52084-e4e7-4bf0-b27f-49ad3821297c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:38.655521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a52084-e4e7-4bf0-b27f-49ad3821297c service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:38.665275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1021/2049] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:38.701077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa357eab-a0c9-4ab3-9e33-166ff7ac4d46 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:38.812174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa357eab-a0c9-4ab3-9e33-166ff7ac4d46 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:38.817541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:38.818219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:38.818622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:38.818916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] PATH_INFO: `/v3/users/2ebfde4a905d42389abb42bf33ba426d` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:38.820822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ebfde4a905d42389abb42bf33ba426d)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:38.824022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:38.826089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:39.042758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ebfde4a905d42389abb42bf33ba426d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:39.174920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ebfde4a905d42389abb42bf33ba426d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:39.212075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ebfde4a905d42389abb42bf33ba426d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:39.217453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ebfde4a905d42389abb42bf33ba426d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:39.218257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ebfde4a905d42389abb42bf33ba426d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:39.252358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1029/2050] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:38 2022] DELETE /identity/v3/users/2ebfde4a905d42389abb42bf33ba426d => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:39.254590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ee35e-1f8d-4a5c-8a9e-7baf569a27fb None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:39.255520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ee35e-1f8d-4a5c-8a9e-7baf569a27fb None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:39.255989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ee35e-1f8d-4a5c-8a9e-7baf569a27fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:39.292001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-341ee35e-1f8d-4a5c-8a9e-7baf569a27fb None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:39.293170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-341ee35e-1f8d-4a5c-8a9e-7baf569a27fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:39.440682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1022/2051] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:39.453368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:39.454755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:39.546575 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c6964-d9c5-413a-b94a-625c3de30169 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:39.550874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575423c-2e76-441a-a720-89cebc1e2d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:39.551909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575423c-2e76-441a-a720-89cebc1e2d47 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:39.552486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575423c-2e76-441a-a720-89cebc1e2d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:39.553053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575423c-2e76-441a-a720-89cebc1e2d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:39.568157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575423c-2e76-441a-a720-89cebc1e2d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:39.569538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575423c-2e76-441a-a720-89cebc1e2d47 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:39.581737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1030/2052] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:40.744236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ee35e-1f8d-4a5c-8a9e-7baf569a27fb None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:40.754293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-341ee35e-1f8d-4a5c-8a9e-7baf569a27fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:40.760331 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ee35e-1f8d-4a5c-8a9e-7baf569a27fb None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:40.770915 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:40.771556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:40.771834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:40.772102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] PATH_INFO: `/v3/projects/1137426f1915403fa7b008d6e9b7e20a` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:40.773232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1137426f1915403fa7b008d6e9b7e20a)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:40.775327 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:40.894909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1137426f1915403fa7b008d6e9b7e20a', 'request_id': 'req-7e1ada74-5125-4872-b210-d65da5f06f77'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:40.987123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1137426f1915403fa7b008d6e9b7e20a', 'request_id': 'req-7e1ada74-5125-4872-b210-d65da5f06f77'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:41.002033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1137426f1915403fa7b008d6e9b7e20a', 'request_id': 'req-7e1ada74-5125-4872-b210-d65da5f06f77'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:41.010887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] The token cache is being invalidate because project 1137426f1915403fa7b008d6e9b7e20a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:41.011195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:41.012512 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:41.013924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1023/2053] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:40 2022] DELETE /identity/v3/projects/1137426f1915403fa7b008d6e9b7e20a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:41.018109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575423c-2e76-441a-a720-89cebc1e2d47 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:41.029895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575423c-2e76-441a-a720-89cebc1e2d47 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:41.105864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:41.106870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:41.107131 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:41.107402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] PATH_INFO: `/v3/users/88a2ec6d55914aa4924a4d5c7f104a1e` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:41.108713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] RBAC: Authorizing `identity:delete_user(user_id=88a2ec6d55914aa4924a4d5c7f104a1e)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:41.110670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:41.145812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:41.382348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88a2ec6d55914aa4924a4d5c7f104a1e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:41.457356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88a2ec6d55914aa4924a4d5c7f104a1e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:41.481467 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88a2ec6d55914aa4924a4d5c7f104a1e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:41.484594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88a2ec6d55914aa4924a4d5c7f104a1e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:41.486078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4a491c57-6c9a-49ab-bb06-0199f290e95a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88a2ec6d55914aa4924a4d5c7f104a1e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:41.521528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1031/2054] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:41 2022] DELETE /identity/v3/users/88a2ec6d55914aa4924a4d5c7f104a1e => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:42.092931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:42.095179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:42.124006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ada74-5125-4872-b210-d65da5f06f77 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:42.175797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:42.231134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:42.232501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:42.233028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:42.233463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] PATH_INFO: `/v3/projects/a85c30a1cbf74311a0c42927ced737a5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:42.235023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85c30a1cbf74311a0c42927ced737a5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:42.237128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:42.363204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85c30a1cbf74311a0c42927ced737a5', 'request_id': 'req-58c85df0-e495-40f7-906f-e5084015e8bf'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:42.449675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85c30a1cbf74311a0c42927ced737a5', 'request_id': 'req-58c85df0-e495-40f7-906f-e5084015e8bf'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:42.457671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85c30a1cbf74311a0c42927ced737a5', 'request_id': 'req-58c85df0-e495-40f7-906f-e5084015e8bf'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:42.471203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] The token cache is being invalidate because project a85c30a1cbf74311a0c42927ced737a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:42.471801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:42.472838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-58c85df0-e495-40f7-906f-e5084015e8bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:42.478005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1024/2055] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:42 2022] DELETE /identity/v3/projects/a85c30a1cbf74311a0c42927ced737a5 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:42.480519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba921d10-a68c-482f-aa4e-ea679e59f20b None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:42.483620 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba921d10-a68c-482f-aa4e-ea679e59f20b None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:42.483620 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba921d10-a68c-482f-aa4e-ea679e59f20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:42.504986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ba921d10-a68c-482f-aa4e-ea679e59f20b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:42.507394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ba921d10-a68c-482f-aa4e-ea679e59f20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:42.652100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1032/2056] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:42.666532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773baaaf-1d97-4cc7-acdd-ec3a3d1ccc38 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:42.667148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773baaaf-1d97-4cc7-acdd-ec3a3d1ccc38 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:42.667451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773baaaf-1d97-4cc7-acdd-ec3a3d1ccc38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:42.702820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-773baaaf-1d97-4cc7-acdd-ec3a3d1ccc38 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:42.704389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-773baaaf-1d97-4cc7-acdd-ec3a3d1ccc38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:42.716980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1025/2057] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:42.721045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba921d10-a68c-482f-aa4e-ea679e59f20b None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:42.735285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba921d10-a68c-482f-aa4e-ea679e59f20b None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:42.742827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a7d92-6f96-4d16-9ba8-f6488a185b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:42.743608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a7d92-6f96-4d16-9ba8-f6488a185b4c admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:42.744945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a7d92-6f96-4d16-9ba8-f6488a185b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:42.745324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a7d92-6f96-4d16-9ba8-f6488a185b4c admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:42.746104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a7d92-6f96-4d16-9ba8-f6488a185b4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:42.746528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a7d92-6f96-4d16-9ba8-f6488a185b4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:42.748435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a7d92-6f96-4d16-9ba8-f6488a185b4c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:42.782459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1033/2058] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:53:42.783771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773baaaf-1d97-4cc7-acdd-ec3a3d1ccc38 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:42.793828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773baaaf-1d97-4cc7-acdd-ec3a3d1ccc38 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:42.798586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4bb9e-126c-497a-9172-00a893b87aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:42.799303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4bb9e-126c-497a-9172-00a893b87aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:42.799708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4bb9e-126c-497a-9172-00a893b87aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:42.800066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4bb9e-126c-497a-9172-00a893b87aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:42.800758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4bb9e-126c-497a-9172-00a893b87aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:42.805256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4bb9e-126c-497a-9172-00a893b87aa3 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:42.896235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1026/2059] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:42 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:42.901036 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a7d92-6f96-4d16-9ba8-f6488a185b4c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:42.914023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a7d92-6f96-4d16-9ba8-f6488a185b4c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:43.003996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015334b-ab07-4f14-8fcd-50b0f1d2b67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:43.004806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015334b-ab07-4f14-8fcd-50b0f1d2b67c admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:43.005080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015334b-ab07-4f14-8fcd-50b0f1d2b67c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:43.005346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015334b-ab07-4f14-8fcd-50b0f1d2b67c admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:43.006016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015334b-ab07-4f14-8fcd-50b0f1d2b67c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:43.007979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015334b-ab07-4f14-8fcd-50b0f1d2b67c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:43.129308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1034/2060] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:42 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:43.133106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4bb9e-126c-497a-9172-00a893b87aa3 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:43.143709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4bb9e-126c-497a-9172-00a893b87aa3 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:43.150242 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dbecf-c06d-46d5-bb0a-eede511249a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:43.151668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dbecf-c06d-46d5-bb0a-eede511249a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:43.152199 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dbecf-c06d-46d5-bb0a-eede511249a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:43.152668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dbecf-c06d-46d5-bb0a-eede511249a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:43.153513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9dbecf-c06d-46d5-bb0a-eede511249a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:43.155331 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9dbecf-c06d-46d5-bb0a-eede511249a6 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:43.170539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1027/2061] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:43.171409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015334b-ab07-4f14-8fcd-50b0f1d2b67c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:43.184512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015334b-ab07-4f14-8fcd-50b0f1d2b67c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:43.191469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d67724-5c53-410d-b50d-e38a3ca57dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:43.192043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d67724-5c53-410d-b50d-e38a3ca57dab admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:43.192334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d67724-5c53-410d-b50d-e38a3ca57dab admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:43.192603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d67724-5c53-410d-b50d-e38a3ca57dab admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:43.193241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d67724-5c53-410d-b50d-e38a3ca57dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:43.194898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d67724-5c53-410d-b50d-e38a3ca57dab admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:43.210013 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1035/2062] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:43.211939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dbecf-c06d-46d5-bb0a-eede511249a6 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:43.239808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dbecf-c06d-46d5-bb0a-eede511249a6 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:43.253768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05be6e-4900-4171-a5e0-7408e19e2a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:43.254490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05be6e-4900-4171-a5e0-7408e19e2a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:43.254821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05be6e-4900-4171-a5e0-7408e19e2a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:43.255096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05be6e-4900-4171-a5e0-7408e19e2a47 admin admin] PATH_INFO: `/v3/projects/769be38ba0504c10ba39a63c94269553/users/224d23e2f4ca436db430633ea0216e50/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:43.280980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05be6e-4900-4171-a5e0-7408e19e2a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=769be38ba0504c10ba39a63c94269553, user_id=224d23e2f4ca436db430633ea0216e50, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:43.283943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05be6e-4900-4171-a5e0-7408e19e2a47 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:43.449041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1028/2063] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:43 2022] PUT /identity/v3/projects/769be38ba0504c10ba39a63c94269553/users/224d23e2f4ca436db430633ea0216e50/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:43.459512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2fc7d-5e73-499e-bb48-f1f2b29a64b8 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:43.459818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2fc7d-5e73-499e-bb48-f1f2b29a64b8 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:43.460090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2fc7d-5e73-499e-bb48-f1f2b29a64b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:43.480208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:43.499170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-4ec2fc7d-5e73-499e-bb48-f1f2b29a64b8 None None] MFA Rules not processed for user `224d23e2f4ca436db430633ea0216e50`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:43.501968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4ec2fc7d-5e73-499e-bb48-f1f2b29a64b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:43.590229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1036/2064] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:43.592971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05be6e-4900-4171-a5e0-7408e19e2a47 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:43.672313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:43.685211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05be6e-4900-4171-a5e0-7408e19e2a47 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:43.694437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f30db7-14c8-4d96-8aca-177c6b1f0f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:43.695296 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f30db7-14c8-4d96-8aca-177c6b1f0f20 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:43.695506 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f30db7-14c8-4d96-8aca-177c6b1f0f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:43.695793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f30db7-14c8-4d96-8aca-177c6b1f0f20 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:43.696413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f30db7-14c8-4d96-8aca-177c6b1f0f20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:43.704861 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f30db7-14c8-4d96-8aca-177c6b1f0f20 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:43.827845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1029/2065] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:43 2022] POST /identity/v3/projects => generated 383 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:43.831683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2fc7d-5e73-499e-bb48-f1f2b29a64b8 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:43.845489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2fc7d-5e73-499e-bb48-f1f2b29a64b8 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:43.949998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa349446-98ca-45f6-b21f-d3a8e06a492d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:43.949998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa349446-98ca-45f6-b21f-d3a8e06a492d admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:43.949998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa349446-98ca-45f6-b21f-d3a8e06a492d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:43.949998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa349446-98ca-45f6-b21f-d3a8e06a492d admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:43.950579 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa349446-98ca-45f6-b21f-d3a8e06a492d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:43.952972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa349446-98ca-45f6-b21f-d3a8e06a492d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:44.081364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1037/2066] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:43 2022] POST /identity/v3/users => generated 298 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:44.082988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f30db7-14c8-4d96-8aca-177c6b1f0f20 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:44.094362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f30db7-14c8-4d96-8aca-177c6b1f0f20 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:44.100473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fc390-8a52-4102-ba10-3fe7364c1f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:44.101212 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fc390-8a52-4102-ba10-3fe7364c1f11 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:44.101727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fc390-8a52-4102-ba10-3fe7364c1f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:44.102211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140fc390-8a52-4102-ba10-3fe7364c1f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:44.103003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fc390-8a52-4102-ba10-3fe7364c1f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:44.105256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140fc390-8a52-4102-ba10-3fe7364c1f11 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:44.116979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1030/2067] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 16:53:44.122999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa349446-98ca-45f6-b21f-d3a8e06a492d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:44.145991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa349446-98ca-45f6-b21f-d3a8e06a492d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:44.148978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0913b-1677-45c3-8611-840ae8b5eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:44.148978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0913b-1677-45c3-8611-840ae8b5eb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:44.149138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0913b-1677-45c3-8611-840ae8b5eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:44.149365 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0913b-1677-45c3-8611-840ae8b5eb0a admin admin] PATH_INFO: `/v3/projects/d25d41924acf48e782b0dcb362334f50/users/82cbd6908bbe41a6a4d542235607efbe/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:44.182750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0913b-1677-45c3-8611-840ae8b5eb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25d41924acf48e782b0dcb362334f50, user_id=82cbd6908bbe41a6a4d542235607efbe, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:44.188411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0913b-1677-45c3-8611-840ae8b5eb0a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:44.798276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1038/2068] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:44 2022] PUT /identity/v3/projects/d25d41924acf48e782b0dcb362334f50/users/82cbd6908bbe41a6a4d542235607efbe/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:44.803444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fc390-8a52-4102-ba10-3fe7364c1f11 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:44.815128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140fc390-8a52-4102-ba10-3fe7364c1f11 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:44.885913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae678df-1d44-4b8b-8033-7006c17d3c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:44.906478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae678df-1d44-4b8b-8033-7006c17d3c8f admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:44.907036 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae678df-1d44-4b8b-8033-7006c17d3c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:44.907718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae678df-1d44-4b8b-8033-7006c17d3c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:44.908815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae678df-1d44-4b8b-8033-7006c17d3c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:44.910917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae678df-1d44-4b8b-8033-7006c17d3c8f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:44.927814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1031/2069] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:44.929108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0913b-1677-45c3-8611-840ae8b5eb0a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:44.973837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0913b-1677-45c3-8611-840ae8b5eb0a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:44.996004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c2a34-c7f3-497b-9250-cbce4c60c509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:44.996967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c2a34-c7f3-497b-9250-cbce4c60c509 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:44.997408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c2a34-c7f3-497b-9250-cbce4c60c509 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:44.997935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c2a34-c7f3-497b-9250-cbce4c60c509 admin admin] PATH_INFO: `/v3/projects/d25d41924acf48e782b0dcb362334f50/users/82cbd6908bbe41a6a4d542235607efbe/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:45.004915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c2a34-c7f3-497b-9250-cbce4c60c509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25d41924acf48e782b0dcb362334f50, user_id=82cbd6908bbe41a6a4d542235607efbe, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:45.008622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c2a34-c7f3-497b-9250-cbce4c60c509 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:45.071461 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d460b1a-2a0a-4551-b6ce-8b8026e290db None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:45.071461 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d460b1a-2a0a-4551-b6ce-8b8026e290db None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:45.071739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d460b1a-2a0a-4551-b6ce-8b8026e290db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:45.084103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1039/2070] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:44 2022] PUT /identity/v3/projects/d25d41924acf48e782b0dcb362334f50/users/82cbd6908bbe41a6a4d542235607efbe/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:45.116480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c2a34-c7f3-497b-9250-cbce4c60c509 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:45.116480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-521c2a34-c7f3-497b-9250-cbce4c60c509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:45.147035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-1d460b1a-2a0a-4551-b6ce-8b8026e290db None None] MFA Rules not processed for user `82cbd6908bbe41a6a4d542235607efbe`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:45.159961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1d460b1a-2a0a-4551-b6ce-8b8026e290db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:45.256345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:45.277400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c2a34-c7f3-497b-9250-cbce4c60c509 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:45.284886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:45.285530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:45.285812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:45.286171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] PATH_INFO: `/v3/users/7d77a70ecf2c4657a09caeda776743c4` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:45.294960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d77a70ecf2c4657a09caeda776743c4)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:45.294960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:45.296653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1032/2071] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:45.308246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d460b1a-2a0a-4551-b6ce-8b8026e290db None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:45.313108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1d460b1a-2a0a-4551-b6ce-8b8026e290db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:45.403443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:45.408656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d460b1a-2a0a-4551-b6ce-8b8026e290db None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:45.413186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ab815-cc85-4b6e-b3b5-e7d63ccd1a70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:45.414073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ab815-cc85-4b6e-b3b5-e7d63ccd1a70 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:45.414655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ab815-cc85-4b6e-b3b5-e7d63ccd1a70 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:45.415032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ab815-cc85-4b6e-b3b5-e7d63ccd1a70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:45.435750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ab815-cc85-4b6e-b3b5-e7d63ccd1a70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:45.438136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ab815-cc85-4b6e-b3b5-e7d63ccd1a70 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:45.457241 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1033/2072] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:45.581397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ab815-cc85-4b6e-b3b5-e7d63ccd1a70 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:45.601744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d77a70ecf2c4657a09caeda776743c4'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:45.608898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ab815-cc85-4b6e-b3b5-e7d63ccd1a70 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:45.615339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:45.616126 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:45.616536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:45.616914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] PATH_INFO: `/v3/users/224d23e2f4ca436db430633ea0216e50` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:45.619419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=224d23e2f4ca436db430633ea0216e50)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:45.622747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:45.709397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d77a70ecf2c4657a09caeda776743c4'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:45.757809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d77a70ecf2c4657a09caeda776743c4'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:45.759299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d77a70ecf2c4657a09caeda776743c4'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:45.760553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-9dc35cfb-3aab-457c-acce-bbdacbadc78f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d77a70ecf2c4657a09caeda776743c4'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:45.782290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1040/2073] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:45 2022] DELETE /identity/v3/users/7d77a70ecf2c4657a09caeda776743c4 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:45.786884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc638b1-713a-4767-8800-667fe1336867 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:45.787170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc638b1-713a-4767-8800-667fe1336867 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:45.787438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc638b1-713a-4767-8800-667fe1336867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:45.814831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224d23e2f4ca436db430633ea0216e50'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:45.817472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ebc638b1-713a-4767-8800-667fe1336867 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:45.818728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ebc638b1-713a-4767-8800-667fe1336867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:45.885424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224d23e2f4ca436db430633ea0216e50'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:45.896482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224d23e2f4ca436db430633ea0216e50'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:45.905101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1041/2074] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:45.916332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc638b1-713a-4767-8800-667fe1336867 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:45.918880 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ebc638b1-713a-4767-8800-667fe1336867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:45.923955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224d23e2f4ca436db430633ea0216e50'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:45.925071 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-698bf8f8-4f68-42c0-8170-df90f0b36a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224d23e2f4ca436db430633ea0216e50'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:45.934008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1034/2075] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:45 2022] DELETE /identity/v3/users/224d23e2f4ca436db430633ea0216e50 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:45.938473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288a9ec-99ac-455b-8484-b531f1c6eeea None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:45.938975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288a9ec-99ac-455b-8484-b531f1c6eeea None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:45.939560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5288a9ec-99ac-455b-8484-b531f1c6eeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:45.971045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5288a9ec-99ac-455b-8484-b531f1c6eeea None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:45.972418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5288a9ec-99ac-455b-8484-b531f1c6eeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:46.022014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc638b1-713a-4767-8800-667fe1336867 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:46.044417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:46.059893 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1035/2076] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:46.060806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288a9ec-99ac-455b-8484-b531f1c6eeea None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:46.070794 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5288a9ec-99ac-455b-8484-b531f1c6eeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:46.084474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5288a9ec-99ac-455b-8484-b531f1c6eeea None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:46.091538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:46.093855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:46.093855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:46.093855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] PATH_INFO: `/v3/users/962e349dd4c1469db54adbd7433025df` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:46.093855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] RBAC: Authorizing `identity:delete_user(user_id=962e349dd4c1469db54adbd7433025df)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:46.094708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166487-443b-45d7-9384-676eeb411b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:46.095863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:46.095989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166487-443b-45d7-9384-676eeb411b10 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:46.096336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166487-443b-45d7-9384-676eeb411b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:46.096695 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3166487-443b-45d7-9384-676eeb411b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:46.098571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d3166487-443b-45d7-9384-676eeb411b10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:46.114914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3166487-443b-45d7-9384-676eeb411b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:46.116731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3166487-443b-45d7-9384-676eeb411b10 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:46.128476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1042/2077] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:46.129460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166487-443b-45d7-9384-676eeb411b10 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:46.131307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d3166487-443b-45d7-9384-676eeb411b10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:46.142532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3166487-443b-45d7-9384-676eeb411b10 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:46.147397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e5f97-8526-48f4-bc70-913de480656e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:46.148000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e5f97-8526-48f4-bc70-913de480656e service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:46.148273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e5f97-8526-48f4-bc70-913de480656e service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:46.148539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e5f97-8526-48f4-bc70-913de480656e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:46.160734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e5f97-8526-48f4-bc70-913de480656e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:46.162190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e5f97-8526-48f4-bc70-913de480656e service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:46.172543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1043/2078] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:46.373851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '962e349dd4c1469db54adbd7433025df'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:46.446238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '962e349dd4c1469db54adbd7433025df'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:46.453812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:46.456710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '962e349dd4c1469db54adbd7433025df'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:46.473823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '962e349dd4c1469db54adbd7433025df'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:46.475916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '962e349dd4c1469db54adbd7433025df'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:46.515211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84b047-16ac-48b7-b7aa-67419a2c057a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:46.516398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84b047-16ac-48b7-b7aa-67419a2c057a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:46.518125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f84b047-16ac-48b7-b7aa-67419a2c057a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:46.521751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1036/2079] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:46 2022] DELETE /identity/v3/users/962e349dd4c1469db54adbd7433025df => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:46.552076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9f84b047-16ac-48b7-b7aa-67419a2c057a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:46.555304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9f84b047-16ac-48b7-b7aa-67419a2c057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:46.648222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1044/2080] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:46.658375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:46.659788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:46.674080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84b047-16ac-48b7-b7aa-67419a2c057a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:46.675638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9f84b047-16ac-48b7-b7aa-67419a2c057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:46.696734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f84b047-16ac-48b7-b7aa-67419a2c057a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:46.708181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:46.708982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:46.709368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:46.709722 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] PATH_INFO: `/v3/projects/7b75fb609a4d4df3b23e88d85a79650a` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:46.711401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b75fb609a4d4df3b23e88d85a79650a)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:46.713906 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:46.760823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454a0a2-af11-4a72-a7b9-dc963be5015a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:46.765376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770f379-06f2-430f-8673-8ca8fd13e931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:46.766711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770f379-06f2-430f-8673-8ca8fd13e931 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:46.767901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770f379-06f2-430f-8673-8ca8fd13e931 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:46.768343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770f379-06f2-430f-8673-8ca8fd13e931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:46.797325 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770f379-06f2-430f-8673-8ca8fd13e931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:46.799025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770f379-06f2-430f-8673-8ca8fd13e931 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:46.816770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1037/2081] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:46.819846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770f379-06f2-430f-8673-8ca8fd13e931 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:46.820688 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1770f379-06f2-430f-8673-8ca8fd13e931 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:46.840448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770f379-06f2-430f-8673-8ca8fd13e931 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:46.862083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b75fb609a4d4df3b23e88d85a79650a', 'request_id': 'req-5493cd8e-ea89-44a6-aa41-48fb0ece2709'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:46.917815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:46.918484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:46.918763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:46.919032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] PATH_INFO: `/v3/projects/769be38ba0504c10ba39a63c94269553` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:46.920219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=769be38ba0504c10ba39a63c94269553)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:46.922175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:46.958856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b75fb609a4d4df3b23e88d85a79650a', 'request_id': 'req-5493cd8e-ea89-44a6-aa41-48fb0ece2709'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:46.966935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b75fb609a4d4df3b23e88d85a79650a', 'request_id': 'req-5493cd8e-ea89-44a6-aa41-48fb0ece2709'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:46.979159 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] The token cache is being invalidate because project 7b75fb609a4d4df3b23e88d85a79650a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:46.979708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:46.980640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5493cd8e-ea89-44a6-aa41-48fb0ece2709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:46.982352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1045/2082] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:46 2022] DELETE /identity/v3/projects/7b75fb609a4d4df3b23e88d85a79650a => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:46.988730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8acd22-d512-400d-8b56-1e05b3d756df None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:46.988730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8acd22-d512-400d-8b56-1e05b3d756df None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:46.989372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8acd22-d512-400d-8b56-1e05b3d756df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:47.012154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ec8acd22-d512-400d-8b56-1e05b3d756df None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:47.014181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ec8acd22-d512-400d-8b56-1e05b3d756df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:47.032297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1046/2083] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:47.035574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b79ba-a283-4b4a-8d9f-a6691f61cdec None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:47.035986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b79ba-a283-4b4a-8d9f-a6691f61cdec None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:47.036317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b79ba-a283-4b4a-8d9f-a6691f61cdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:47.084568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d69b79ba-a283-4b4a-8d9f-a6691f61cdec None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:47.088571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d69b79ba-a283-4b4a-8d9f-a6691f61cdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:47.095289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '769be38ba0504c10ba39a63c94269553', 'request_id': 'req-b559e27c-07bc-4528-8b17-a0a79a0ee88a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.121436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:47.159492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '769be38ba0504c10ba39a63c94269553', 'request_id': 'req-b559e27c-07bc-4528-8b17-a0a79a0ee88a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.168342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '769be38ba0504c10ba39a63c94269553', 'request_id': 'req-b559e27c-07bc-4528-8b17-a0a79a0ee88a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.171834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1047/2084] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:47.175707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b79ba-a283-4b4a-8d9f-a6691f61cdec None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:47.199768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] The token cache is being invalidate because project 769be38ba0504c10ba39a63c94269553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:47.199768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.199768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.199768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1038/2085] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:46 2022] DELETE /identity/v3/projects/769be38ba0504c10ba39a63c94269553 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:47.203128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:47.203320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b79ba-a283-4b4a-8d9f-a6691f61cdec None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:47.205608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:47.209570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063db545-4f62-4ca6-ba2f-e56def1e2744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:47.210213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063db545-4f62-4ca6-ba2f-e56def1e2744 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:47.210485 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063db545-4f62-4ca6-ba2f-e56def1e2744 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:47.210919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063db545-4f62-4ca6-ba2f-e56def1e2744 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:47.211579 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063db545-4f62-4ca6-ba2f-e56def1e2744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:47.211984 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063db545-4f62-4ca6-ba2f-e56def1e2744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:47.214197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063db545-4f62-4ca6-ba2f-e56def1e2744 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:47.224289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559e27c-07bc-4528-8b17-a0a79a0ee88a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:47.231843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:47.233217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:47.233779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:47.234570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] PATH_INFO: `/v3/users/82cbd6908bbe41a6a4d542235607efbe` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:47.236613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82cbd6908bbe41a6a4d542235607efbe)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:47.239673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1048/2086] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:53:47.239827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:47.243601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:47.244962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063db545-4f62-4ca6-ba2f-e56def1e2744 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:47.247850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-063db545-4f62-4ca6-ba2f-e56def1e2744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:47.263957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063db545-4f62-4ca6-ba2f-e56def1e2744 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:47.268980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55208e8-1a5c-492d-938d-86b3c4bca4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:47.269600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55208e8-1a5c-492d-938d-86b3c4bca4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:47.269877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55208e8-1a5c-492d-938d-86b3c4bca4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:47.270215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55208e8-1a5c-492d-938d-86b3c4bca4e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:47.270842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55208e8-1a5c-492d-938d-86b3c4bca4e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:47.272989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55208e8-1a5c-492d-938d-86b3c4bca4e0 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:47.425429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1049/2087] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:47 2022] POST /identity/v3/projects => generated 381 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:47.430065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55208e8-1a5c-492d-938d-86b3c4bca4e0 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:47.568048 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82cbd6908bbe41a6a4d542235607efbe'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.579789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55208e8-1a5c-492d-938d-86b3c4bca4e0 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:47.585995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dc457-ddbb-4a89-86ef-bbf1ff1d20c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:47.586724 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc457-ddbb-4a89-86ef-bbf1ff1d20c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:47.587076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc457-ddbb-4a89-86ef-bbf1ff1d20c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:47.587408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dc457-ddbb-4a89-86ef-bbf1ff1d20c9 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:47.588965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dc457-ddbb-4a89-86ef-bbf1ff1d20c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:47.592121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dc457-ddbb-4a89-86ef-bbf1ff1d20c9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:47.696550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82cbd6908bbe41a6a4d542235607efbe'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.709876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1050/2088] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:47 2022] POST /identity/v3/users => generated 291 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:47.709876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dc457-ddbb-4a89-86ef-bbf1ff1d20c9 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:47.709876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-948dc457-ddbb-4a89-86ef-bbf1ff1d20c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:47.714796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82cbd6908bbe41a6a4d542235607efbe'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.729605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dc457-ddbb-4a89-86ef-bbf1ff1d20c9 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:47.733310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82cbd6908bbe41a6a4d542235607efbe'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.734155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82cbd6908bbe41a6a4d542235607efbe'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.736018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:47.739568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:47.740470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:47.740979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] PATH_INFO: `/v3/projects/54c43bdef3b24686b28b10b1581becae` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:47.742568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c43bdef3b24686b28b10b1581becae)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:47.745829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:47.748083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1039/2089] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:47 2022] DELETE /identity/v3/users/82cbd6908bbe41a6a4d542235607efbe => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:47.751106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:47.761737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:47.852013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e40464-cae5-40e9-b460-cd233b9ece7a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:47.859143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43c04a-e4da-46bf-807f-831f78da6ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:47.863610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43c04a-e4da-46bf-807f-831f78da6ecf admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:47.863610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43c04a-e4da-46bf-807f-831f78da6ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:47.863610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43c04a-e4da-46bf-807f-831f78da6ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:47.863610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43c04a-e4da-46bf-807f-831f78da6ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:47.864595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43c04a-e4da-46bf-807f-831f78da6ecf admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:47.877764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1040/2090] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:47.883028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43c04a-e4da-46bf-807f-831f78da6ecf admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:47.905669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c43bdef3b24686b28b10b1581becae', 'request_id': 'req-50be6e88-ef13-46b6-aac8-fcdc99463ec5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:47.908426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43c04a-e4da-46bf-807f-831f78da6ecf admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:47.986630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb4474-f24d-4e1c-b2dc-35f1023228c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:47.987254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb4474-f24d-4e1c-b2dc-35f1023228c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:47.987534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb4474-f24d-4e1c-b2dc-35f1023228c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:47.988264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb4474-f24d-4e1c-b2dc-35f1023228c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:47.988653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb4474-f24d-4e1c-b2dc-35f1023228c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:47.990412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb4474-f24d-4e1c-b2dc-35f1023228c9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:48.006373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1041/2091] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:48.008033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb4474-f24d-4e1c-b2dc-35f1023228c9 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:48.015070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c43bdef3b24686b28b10b1581becae', 'request_id': 'req-50be6e88-ef13-46b6-aac8-fcdc99463ec5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:48.023909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb4474-f24d-4e1c-b2dc-35f1023228c9 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:48.031807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05187e-3e6a-4332-b0d5-d0cdba4c2f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:48.032457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05187e-3e6a-4332-b0d5-d0cdba4c2f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:48.033266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05187e-3e6a-4332-b0d5-d0cdba4c2f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:48.033810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05187e-3e6a-4332-b0d5-d0cdba4c2f66 admin admin] PATH_INFO: `/v3/projects/883aca80f1f14b80b8080fe9d72e3215/users/f493a8495dd64a94b12e159862167408/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:48.036997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c43bdef3b24686b28b10b1581becae', 'request_id': 'req-50be6e88-ef13-46b6-aac8-fcdc99463ec5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:48.058768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:48.062892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] The token cache is being invalidate because project 54c43bdef3b24686b28b10b1581becae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:48.063365 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:48.065839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05187e-3e6a-4332-b0d5-d0cdba4c2f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=883aca80f1f14b80b8080fe9d72e3215, user_id=f493a8495dd64a94b12e159862167408, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:48.066550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-50be6e88-ef13-46b6-aac8-fcdc99463ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:48.066973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1051/2092] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:47 2022] DELETE /identity/v3/projects/54c43bdef3b24686b28b10b1581becae => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:48.068075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05187e-3e6a-4332-b0d5-d0cdba4c2f66 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:48.160560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1042/2093] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:48 2022] PUT /identity/v3/projects/883aca80f1f14b80b8080fe9d72e3215/users/f493a8495dd64a94b12e159862167408/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:48.166660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40458b1c-385f-443c-a6b0-e0be4131f26c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:48.167268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40458b1c-385f-443c-a6b0-e0be4131f26c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:48.168038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40458b1c-385f-443c-a6b0-e0be4131f26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:48.215240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-40458b1c-385f-443c-a6b0-e0be4131f26c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:48.216594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-40458b1c-385f-443c-a6b0-e0be4131f26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:48.234481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:48.318114 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1052/2094] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:48.331744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05187e-3e6a-4332-b0d5-d0cdba4c2f66 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:48.334318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2f05187e-3e6a-4332-b0d5-d0cdba4c2f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:48.418100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05187e-3e6a-4332-b0d5-d0cdba4c2f66 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:48.434995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4c35f-fa7e-4822-88d8-74dffd1ae0d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:48.434995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4c35f-fa7e-4822-88d8-74dffd1ae0d5 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:48.434995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4c35f-fa7e-4822-88d8-74dffd1ae0d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:48.434995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4c35f-fa7e-4822-88d8-74dffd1ae0d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:48.450302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4c35f-fa7e-4822-88d8-74dffd1ae0d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:48.452056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4c35f-fa7e-4822-88d8-74dffd1ae0d5 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:48.465858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1043/2095] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:48.510993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40458b1c-385f-443c-a6b0-e0be4131f26c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:48.512641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-40458b1c-385f-443c-a6b0-e0be4131f26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:48.529461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40458b1c-385f-443c-a6b0-e0be4131f26c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:48.534910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:48.535792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:48.536287 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:48.536710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] PATH_INFO: `/v3/projects/d25d41924acf48e782b0dcb362334f50` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:48.538910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d25d41924acf48e782b0dcb362334f50)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:48.541038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:48.798942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd25d41924acf48e782b0dcb362334f50', 'request_id': 'req-537c3877-7dc7-4d66-af49-004f4cf609cd'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:49.420382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd25d41924acf48e782b0dcb362334f50', 'request_id': 'req-537c3877-7dc7-4d66-af49-004f4cf609cd'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:49.431418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd25d41924acf48e782b0dcb362334f50', 'request_id': 'req-537c3877-7dc7-4d66-af49-004f4cf609cd'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:49.448966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] The token cache is being invalidate because project d25d41924acf48e782b0dcb362334f50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:49.449326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:49.450103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-537c3877-7dc7-4d66-af49-004f4cf609cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:49.453088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1053/2096] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:48 2022] DELETE /identity/v3/projects/d25d41924acf48e782b0dcb362334f50 => generated 0 bytes in 944 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:49.458396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba47860-ca9c-4c62-9b19-096554fbb394 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:49.458396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba47860-ca9c-4c62-9b19-096554fbb394 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:49.458396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba47860-ca9c-4c62-9b19-096554fbb394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:49.481783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9ba47860-ca9c-4c62-9b19-096554fbb394 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:49.486697 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9ba47860-ca9c-4c62-9b19-096554fbb394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:49.565990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1044/2097] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:49.570097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27ffcf-c0e7-45c8-a881-0b1ed53165ce None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:49.570387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27ffcf-c0e7-45c8-a881-0b1ed53165ce None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:49.570659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27ffcf-c0e7-45c8-a881-0b1ed53165ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:49.610430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-dc27ffcf-c0e7-45c8-a881-0b1ed53165ce None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:49.611778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-dc27ffcf-c0e7-45c8-a881-0b1ed53165ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:49.632576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1054/2098] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:49.633188 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba47860-ca9c-4c62-9b19-096554fbb394 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:49.650026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba47860-ca9c-4c62-9b19-096554fbb394 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:49.658263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88836c-a2e2-475d-89af-e9002ce7482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:49.659245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88836c-a2e2-475d-89af-e9002ce7482c admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:49.659722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88836c-a2e2-475d-89af-e9002ce7482c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:49.660243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88836c-a2e2-475d-89af-e9002ce7482c admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:49.661085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88836c-a2e2-475d-89af-e9002ce7482c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:49.661745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88836c-a2e2-475d-89af-e9002ce7482c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:49.663619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88836c-a2e2-475d-89af-e9002ce7482c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:49.679283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1045/2099] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:53:49.683839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27ffcf-c0e7-45c8-a881-0b1ed53165ce None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:49.710573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27ffcf-c0e7-45c8-a881-0b1ed53165ce None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:49.724418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a357af-487a-4476-b737-dfc1347cc316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:49.725040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a357af-487a-4476-b737-dfc1347cc316 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:49.725336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a357af-487a-4476-b737-dfc1347cc316 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:49.725675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a357af-487a-4476-b737-dfc1347cc316 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:49.726339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a357af-487a-4476-b737-dfc1347cc316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:49.728905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a357af-487a-4476-b737-dfc1347cc316 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:49.808667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1055/2100] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:49 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:53:49.813580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88836c-a2e2-475d-89af-e9002ce7482c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:49.828682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88836c-a2e2-475d-89af-e9002ce7482c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:49.906020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fefd83-b143-4848-bf21-32a8e89339e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:49.907075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fefd83-b143-4848-bf21-32a8e89339e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:49.907485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fefd83-b143-4848-bf21-32a8e89339e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:49.908071 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fefd83-b143-4848-bf21-32a8e89339e5 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:49.922165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fefd83-b143-4848-bf21-32a8e89339e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:49.924215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fefd83-b143-4848-bf21-32a8e89339e5 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:50.069160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1046/2101] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:49 2022] POST /identity/v3/users => generated 291 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:50.073679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a357af-487a-4476-b737-dfc1347cc316 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:50.085108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a357af-487a-4476-b737-dfc1347cc316 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:50.089979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6152b-1097-4a43-a723-2afc1ee6bd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:50.090806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6152b-1097-4a43-a723-2afc1ee6bd2c admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:50.091140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6152b-1097-4a43-a723-2afc1ee6bd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:50.091415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6152b-1097-4a43-a723-2afc1ee6bd2c admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:50.092058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6152b-1097-4a43-a723-2afc1ee6bd2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:50.093654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6152b-1097-4a43-a723-2afc1ee6bd2c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:50.105726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1056/2102] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:50.109994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fefd83-b143-4848-bf21-32a8e89339e5 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:50.120987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fefd83-b143-4848-bf21-32a8e89339e5 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:50.125735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745842f-bb01-4710-a3e1-32e9b6ac620f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:50.126347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745842f-bb01-4710-a3e1-32e9b6ac620f admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:50.126621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745842f-bb01-4710-a3e1-32e9b6ac620f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:50.126888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745842f-bb01-4710-a3e1-32e9b6ac620f admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:50.127521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745842f-bb01-4710-a3e1-32e9b6ac620f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:50.129088 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745842f-bb01-4710-a3e1-32e9b6ac620f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:50.131341 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:50.140684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1047/2103] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:50.145555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6152b-1097-4a43-a723-2afc1ee6bd2c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:50.157092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6152b-1097-4a43-a723-2afc1ee6bd2c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:50.161617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbd6fe-5099-49da-b84b-8322b786ee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:50.162306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbd6fe-5099-49da-b84b-8322b786ee99 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:50.162577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbd6fe-5099-49da-b84b-8322b786ee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:50.162844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcbd6fe-5099-49da-b84b-8322b786ee99 admin admin] PATH_INFO: `/v3/projects/f392f37e2df1446c999502bf9fe4b9e7/users/099f691cc49a4cfea462103347a98a3f/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:50.180365 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbd6fe-5099-49da-b84b-8322b786ee99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f392f37e2df1446c999502bf9fe4b9e7, user_id=099f691cc49a4cfea462103347a98a3f, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:50.183689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcbd6fe-5099-49da-b84b-8322b786ee99 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:50.364002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1057/2104] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:50 2022] PUT /identity/v3/projects/f392f37e2df1446c999502bf9fe4b9e7/users/099f691cc49a4cfea462103347a98a3f/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:50.374654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5151791-b8e1-492b-9f0a-b0a3be35f194 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:50.374935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5151791-b8e1-492b-9f0a-b0a3be35f194 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:50.375200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5151791-b8e1-492b-9f0a-b0a3be35f194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:50.401733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-b5151791-b8e1-492b-9f0a-b0a3be35f194 None None] MFA Rules not processed for user `099f691cc49a4cfea462103347a98a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:50.402954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b5151791-b8e1-492b-9f0a-b0a3be35f194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:50.468777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1048/2105] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:50.473692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbd6fe-5099-49da-b84b-8322b786ee99 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:50.562182 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:50.597609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcbd6fe-5099-49da-b84b-8322b786ee99 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:50.603092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb28934-2cf5-49a3-8605-a1d6a3388791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:50.604330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb28934-2cf5-49a3-8605-a1d6a3388791 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:50.604606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb28934-2cf5-49a3-8605-a1d6a3388791 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:50.604927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb28934-2cf5-49a3-8605-a1d6a3388791 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:50.605555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb28934-2cf5-49a3-8605-a1d6a3388791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:50.618964 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb28934-2cf5-49a3-8605-a1d6a3388791 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:50.740265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b628b8-5bd9-4050-94b8-e253550d5656 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:50.740619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b628b8-5bd9-4050-94b8-e253550d5656 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:50.740898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b628b8-5bd9-4050-94b8-e253550d5656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:50.744028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1058/2106] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:50 2022] POST /identity/v3/projects => generated 379 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:50.750910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb28934-2cf5-49a3-8605-a1d6a3388791 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:50.783037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-e9b628b8-5bd9-4050-94b8-e253550d5656 None None] MFA Rules not processed for user `f493a8495dd64a94b12e159862167408`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:50.785034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e9b628b8-5bd9-4050-94b8-e253550d5656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:50.853210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb28934-2cf5-49a3-8605-a1d6a3388791 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:50.859323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82188f46-40ee-4d7e-abd3-8de84fd32b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:50.859938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82188f46-40ee-4d7e-abd3-8de84fd32b66 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:50.860215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82188f46-40ee-4d7e-abd3-8de84fd32b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:50.860487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82188f46-40ee-4d7e-abd3-8de84fd32b66 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:50.861111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82188f46-40ee-4d7e-abd3-8de84fd32b66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:50.863060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82188f46-40ee-4d7e-abd3-8de84fd32b66 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:50.888898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1049/2107] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:50.891460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3286f-b5f8-41a4-819c-762eabb54479 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:50.891825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3286f-b5f8-41a4-819c-762eabb54479 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:50.892392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3286f-b5f8-41a4-819c-762eabb54479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:50.912645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d7b3286f-b5f8-41a4-819c-762eabb54479 None None] MFA Rules not processed for user `8d77f31d797448b6ad4964fa96e2d492`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:50.913863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d7b3286f-b5f8-41a4-819c-762eabb54479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:50.963853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1059/2108] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:50 2022] POST /identity/v3/users => generated 296 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:50.965224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82188f46-40ee-4d7e-abd3-8de84fd32b66 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:50.967530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-82188f46-40ee-4d7e-abd3-8de84fd32b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:50.980661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1050/2109] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:50.981547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3286f-b5f8-41a4-819c-762eabb54479 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:50.992932 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3286f-b5f8-41a4-819c-762eabb54479 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:51.003397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0190e-88f3-497f-8516-57bc1fdcbdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:51.004050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0190e-88f3-497f-8516-57bc1fdcbdcd admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:51.004876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0190e-88f3-497f-8516-57bc1fdcbdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:51.005151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0190e-88f3-497f-8516-57bc1fdcbdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:51.006886 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0190e-88f3-497f-8516-57bc1fdcbdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:51.008880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0190e-88f3-497f-8516-57bc1fdcbdcd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:51.022268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1051/2110] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:51.027091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0190e-88f3-497f-8516-57bc1fdcbdcd admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:51.042109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0190e-88f3-497f-8516-57bc1fdcbdcd admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:51.047783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0133b-33d1-4284-b697-3ae7c384c383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:51.048439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0133b-33d1-4284-b697-3ae7c384c383 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:51.048720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0133b-33d1-4284-b697-3ae7c384c383 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:51.049047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0133b-33d1-4284-b697-3ae7c384c383 admin admin] PATH_INFO: `/v3/projects/ef0e0d543243484796a834e0658867a8/users/f83f3e55c9b44df9ad77f77689a3de01/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:51.059065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82188f46-40ee-4d7e-abd3-8de84fd32b66 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:51.060537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:51.063457 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764eaf00-494a-4c97-8c74-e7ce94961d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:51.064096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764eaf00-494a-4c97-8c74-e7ce94961d2f service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:51.064411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764eaf00-494a-4c97-8c74-e7ce94961d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:51.064714 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764eaf00-494a-4c97-8c74-e7ce94961d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:51.068036 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0133b-33d1-4284-b697-3ae7c384c383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0e0d543243484796a834e0658867a8, user_id=f83f3e55c9b44df9ad77f77689a3de01, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:51.070655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0133b-33d1-4284-b697-3ae7c384c383 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:51.079202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764eaf00-494a-4c97-8c74-e7ce94961d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:51.079202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764eaf00-494a-4c97-8c74-e7ce94961d2f service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:51.089658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1060/2111] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:51.161232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1052/2112] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:51 2022] PUT /identity/v3/projects/ef0e0d543243484796a834e0658867a8/users/f83f3e55c9b44df9ad77f77689a3de01/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:51.165183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764eaf00-494a-4c97-8c74-e7ce94961d2f service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:51.178778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764eaf00-494a-4c97-8c74-e7ce94961d2f service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:51.237199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:51.254615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004ec41-6916-4062-a966-3db2cb0a41fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:51.255224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004ec41-6916-4062-a966-3db2cb0a41fc admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:51.255521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004ec41-6916-4062-a966-3db2cb0a41fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:51.255888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004ec41-6916-4062-a966-3db2cb0a41fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:51.256533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004ec41-6916-4062-a966-3db2cb0a41fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:51.258283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004ec41-6916-4062-a966-3db2cb0a41fc admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:51.269401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1061/2113] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:51.273645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0133b-33d1-4284-b697-3ae7c384c383 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:51.284305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0133b-33d1-4284-b697-3ae7c384c383 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:51.291070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b70430-71be-42d5-aa0b-ec99e96b10f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:51.291843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b70430-71be-42d5-aa0b-ec99e96b10f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:51.294566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b70430-71be-42d5-aa0b-ec99e96b10f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:51.294566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b70430-71be-42d5-aa0b-ec99e96b10f6 admin admin] PATH_INFO: `/v3/projects/ef0e0d543243484796a834e0658867a8/users/f83f3e55c9b44df9ad77f77689a3de01/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:51.294931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b70430-71be-42d5-aa0b-ec99e96b10f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0e0d543243484796a834e0658867a8, user_id=f83f3e55c9b44df9ad77f77689a3de01, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:51.298020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b70430-71be-42d5-aa0b-ec99e96b10f6 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:51.310111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1053/2114] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:51 2022] PUT /identity/v3/projects/ef0e0d543243484796a834e0658867a8/users/f83f3e55c9b44df9ad77f77689a3de01/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 16:53:51.322253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e10838-8c60-4716-856f-376f0579e17b None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:51.323136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e10838-8c60-4716-856f-376f0579e17b None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:51.323136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e10838-8c60-4716-856f-376f0579e17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:51.353779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-47e10838-8c60-4716-856f-376f0579e17b None None] MFA Rules not processed for user `f83f3e55c9b44df9ad77f77689a3de01`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:51.354961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-47e10838-8c60-4716-856f-376f0579e17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:51.431840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1062/2115] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:51.801030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b70430-71be-42d5-aa0b-ec99e96b10f6 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:51.802896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-79b70430-71be-42d5-aa0b-ec99e96b10f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:51.874110 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b70430-71be-42d5-aa0b-ec99e96b10f6 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:51.878047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02408bb5-a1ee-409f-9ae4-77c06c91ac1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:51.878652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02408bb5-a1ee-409f-9ae4-77c06c91ac1f service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:51.878928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02408bb5-a1ee-409f-9ae4-77c06c91ac1f service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:51.879238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02408bb5-a1ee-409f-9ae4-77c06c91ac1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:51.889536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02408bb5-a1ee-409f-9ae4-77c06c91ac1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:51.890839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02408bb5-a1ee-409f-9ae4-77c06c91ac1f service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:51.903052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1054/2116] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:52.506793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e10838-8c60-4716-856f-376f0579e17b None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:52.577074 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e10838-8c60-4716-856f-376f0579e17b None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:52.581756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:52.582401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:52.582680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:52.582947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] PATH_INFO: `/v3/users/099f691cc49a4cfea462103347a98a3f` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:52.584149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] RBAC: Authorizing `identity:delete_user(user_id=099f691cc49a4cfea462103347a98a3f)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:52.586082 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:53.072133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099f691cc49a4cfea462103347a98a3f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:53.181915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099f691cc49a4cfea462103347a98a3f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:53.194836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:53.196156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099f691cc49a4cfea462103347a98a3f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:53.196933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099f691cc49a4cfea462103347a98a3f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:53.197551 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099f691cc49a4cfea462103347a98a3f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:53.209589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1063/2117] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:52 2022] DELETE /identity/v3/users/099f691cc49a4cfea462103347a98a3f => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:53.215362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7870f-97ea-428a-b78b-5307f4093cbb None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:53.215362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7870f-97ea-428a-b78b-5307f4093cbb None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:53.215362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7870f-97ea-428a-b78b-5307f4093cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:53.234089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a8a7870f-97ea-428a-b78b-5307f4093cbb None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:53.235278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a8a7870f-97ea-428a-b78b-5307f4093cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:53.294787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1055/2118] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:53.304728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:53.306307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:53.364941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1c202-1522-444d-8c1a-67ed62f6032f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:53.368578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eda2c2-7992-43da-8279-b6b62acff3dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:53.369195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eda2c2-7992-43da-8279-b6b62acff3dd service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:53.369471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eda2c2-7992-43da-8279-b6b62acff3dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:53.369738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eda2c2-7992-43da-8279-b6b62acff3dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:53.379050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eda2c2-7992-43da-8279-b6b62acff3dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:53.389793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eda2c2-7992-43da-8279-b6b62acff3dd service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:53.390032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1064/2119] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:53:53.850069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7870f-97ea-428a-b78b-5307f4093cbb None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:53.851484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a8a7870f-97ea-428a-b78b-5307f4093cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:53.864783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7870f-97ea-428a-b78b-5307f4093cbb None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:53.869734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:53.870490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:53.870828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:53.871100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] PATH_INFO: `/v3/projects/f392f37e2df1446c999502bf9fe4b9e7` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:53.872237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f392f37e2df1446c999502bf9fe4b9e7)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:53.874162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:53.980289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f392f37e2df1446c999502bf9fe4b9e7', 'request_id': 'req-081cb963-556e-4bbf-9125-8b54b2451053'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:54.070433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f392f37e2df1446c999502bf9fe4b9e7', 'request_id': 'req-081cb963-556e-4bbf-9125-8b54b2451053'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:54.079024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f392f37e2df1446c999502bf9fe4b9e7', 'request_id': 'req-081cb963-556e-4bbf-9125-8b54b2451053'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:54.094001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:54.098749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] The token cache is being invalidate because project f392f37e2df1446c999502bf9fe4b9e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:54.102669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:54.102669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-081cb963-556e-4bbf-9125-8b54b2451053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:54.102669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1056/2120] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:53 2022] DELETE /identity/v3/projects/f392f37e2df1446c999502bf9fe4b9e7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:54.105497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eda2c2-7992-43da-8279-b6b62acff3dd service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:54.116033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eda2c2-7992-43da-8279-b6b62acff3dd service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:54.145026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f589b1f-bcb0-4c05-915d-8a103885d9a3 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:54.148701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f589b1f-bcb0-4c05-915d-8a103885d9a3 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:54.148701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f589b1f-bcb0-4c05-915d-8a103885d9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:54.165647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-1f589b1f-bcb0-4c05-915d-8a103885d9a3 None None] MFA Rules not processed for user `f493a8495dd64a94b12e159862167408`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:54.166930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1f589b1f-bcb0-4c05-915d-8a103885d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:54.190592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:54.191352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:54.191665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:54.191991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] PATH_INFO: `/v3/users/f83f3e55c9b44df9ad77f77689a3de01` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:54.193494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f83f3e55c9b44df9ad77f77689a3de01)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:54.195744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:54.230590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1057/2121] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:54.540076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f83f3e55c9b44df9ad77f77689a3de01'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:54.694330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f83f3e55c9b44df9ad77f77689a3de01'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:54.724670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f83f3e55c9b44df9ad77f77689a3de01'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:54.724670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f83f3e55c9b44df9ad77f77689a3de01'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:54.724670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-46013cc6-23fe-4a21-9ebd-148aa5311da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f83f3e55c9b44df9ad77f77689a3de01'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:54.738376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1065/2122] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:53:54 2022] DELETE /identity/v3/users/f83f3e55c9b44df9ad77f77689a3de01 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:55.532680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f589b1f-bcb0-4c05-915d-8a103885d9a3 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:55.537031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1f589b1f-bcb0-4c05-915d-8a103885d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:55.570793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f589b1f-bcb0-4c05-915d-8a103885d9a3 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:55.661462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:55.662238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:55.662574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:55.662922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] PATH_INFO: `/v3/projects/ef0e0d543243484796a834e0658867a8` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:55.664308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef0e0d543243484796a834e0658867a8)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:55.666368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:55.802277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef0e0d543243484796a834e0658867a8', 'request_id': 'req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:55.873992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6144311-f268-496d-a649-14c8774d3aed None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:55.874867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6144311-f268-496d-a649-14c8774d3aed None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:55.875302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6144311-f268-496d-a649-14c8774d3aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:55.885211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef0e0d543243484796a834e0658867a8', 'request_id': 'req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:55.897397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef0e0d543243484796a834e0658867a8', 'request_id': 'req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:55.901729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c6144311-f268-496d-a649-14c8774d3aed None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:55.903323 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c6144311-f268-496d-a649-14c8774d3aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:55.909992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:55.919643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] The token cache is being invalidate because project ef0e0d543243484796a834e0658867a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:53:55.919643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:55.919643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e1047770-8d5f-4f08-9fa6-0d6b3725a5b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:53:55.919643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1058/2123] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:53:55 2022] DELETE /identity/v3/projects/ef0e0d543243484796a834e0658867a8 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:55.921100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf3a35-6fa7-4417-b433-3c4352bf29a7 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:55.921382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf3a35-6fa7-4417-b433-3c4352bf29a7 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:55.921652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf3a35-6fa7-4417-b433-3c4352bf29a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:55.964671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a0cf3a35-6fa7-4417-b433-3c4352bf29a7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:55.966751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a0cf3a35-6fa7-4417-b433-3c4352bf29a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:56.005369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1066/2124] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 16:53:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:56.013525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6144311-f268-496d-a649-14c8774d3aed None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:56.015143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c6144311-f268-496d-a649-14c8774d3aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:56.018875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:56.049906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1059/2125] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:53:56.063581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8765a7-87af-43d3-9729-959a1946b2fd None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:56.064663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8765a7-87af-43d3-9729-959a1946b2fd None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:56.064958 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8765a7-87af-43d3-9729-959a1946b2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:56.093187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ca8765a7-87af-43d3-9729-959a1946b2fd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:56.098279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ca8765a7-87af-43d3-9729-959a1946b2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:56.109501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1060/2126] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:56.111132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6144311-f268-496d-a649-14c8774d3aed None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:56.119778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8765a7-87af-43d3-9729-959a1946b2fd None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:56.128248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db7e1a-3359-41f6-845e-ea4bf8ee67d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:56.129259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db7e1a-3359-41f6-845e-ea4bf8ee67d7 service swift] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:56.129533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db7e1a-3359-41f6-845e-ea4bf8ee67d7 service swift] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:56.129813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db7e1a-3359-41f6-845e-ea4bf8ee67d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:56.130307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8765a7-87af-43d3-9729-959a1946b2fd None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:56.139175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c8929-a788-4681-a820-4d89ed190a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:56.139895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c8929-a788-4681-a820-4d89ed190a25 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:56.140486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c8929-a788-4681-a820-4d89ed190a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:56.140486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c8929-a788-4681-a820-4d89ed190a25 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:56.141297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c8929-a788-4681-a820-4d89ed190a25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:53:56.141657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c8929-a788-4681-a820-4d89ed190a25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:56.142736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db7e1a-3359-41f6-845e-ea4bf8ee67d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:56.143736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c8929-a788-4681-a820-4d89ed190a25 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:56.144438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db7e1a-3359-41f6-845e-ea4bf8ee67d7 service swift] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:56.158724 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1067/2127] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 16:53:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 16:53:56.158919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1061/2128] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:53:56.162934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db7e1a-3359-41f6-845e-ea4bf8ee67d7 service swift] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:56.185623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db7e1a-3359-41f6-845e-ea4bf8ee67d7 service swift] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:56.191113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566d67-2fbe-4871-ab83-df2ce3114826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:56.195477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566d67-2fbe-4871-ab83-df2ce3114826 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:56.195477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566d67-2fbe-4871-ab83-df2ce3114826 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:56.195477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77566d67-2fbe-4871-ab83-df2ce3114826 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:56.195477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77566d67-2fbe-4871-ab83-df2ce3114826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:56.195477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77566d67-2fbe-4871-ab83-df2ce3114826 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:56.297124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1068/2129] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:53:56 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:56.298644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c8929-a788-4681-a820-4d89ed190a25 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:56.314953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c8929-a788-4681-a820-4d89ed190a25 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:56.472767 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ba874-f135-422a-a40a-aa112bcb576c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:56.474437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ba874-f135-422a-a40a-aa112bcb576c admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:56.474948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ba874-f135-422a-a40a-aa112bcb576c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:56.475449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ba874-f135-422a-a40a-aa112bcb576c admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:56.476672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ba874-f135-422a-a40a-aa112bcb576c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:56.484667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ba874-f135-422a-a40a-aa112bcb576c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:56.630989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1062/2130] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:53:56 2022] POST /identity/v3/users => generated 288 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:53:56.637011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566d67-2fbe-4871-ab83-df2ce3114826 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:56.656814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77566d67-2fbe-4871-ab83-df2ce3114826 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:56.666511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bcdd4-19b8-44bb-b13f-2e5c569de7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:56.667141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bcdd4-19b8-44bb-b13f-2e5c569de7ca admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:56.667576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bcdd4-19b8-44bb-b13f-2e5c569de7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:56.667856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961bcdd4-19b8-44bb-b13f-2e5c569de7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:56.668500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bcdd4-19b8-44bb-b13f-2e5c569de7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:56.670205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961bcdd4-19b8-44bb-b13f-2e5c569de7ca admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:56.694198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1069/2131] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:56.696251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ba874-f135-422a-a40a-aa112bcb576c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:56.717941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ba874-f135-422a-a40a-aa112bcb576c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:56.733540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958316f-b3f0-4d7e-808c-fd941d6a25c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:56.735388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958316f-b3f0-4d7e-808c-fd941d6a25c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:56.735871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958316f-b3f0-4d7e-808c-fd941d6a25c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:56.736276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958316f-b3f0-4d7e-808c-fd941d6a25c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:56.737428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958316f-b3f0-4d7e-808c-fd941d6a25c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:56.742715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958316f-b3f0-4d7e-808c-fd941d6a25c4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:56.766328 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1063/2132] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:53:56.770472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bcdd4-19b8-44bb-b13f-2e5c569de7ca admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:56.784718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961bcdd4-19b8-44bb-b13f-2e5c569de7ca admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:56.799823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7683ec7-4d3a-4d22-ab18-4ef0f4cd3267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:56.800576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7683ec7-4d3a-4d22-ab18-4ef0f4cd3267 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:56.800855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7683ec7-4d3a-4d22-ab18-4ef0f4cd3267 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:56.803021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7683ec7-4d3a-4d22-ab18-4ef0f4cd3267 admin admin] PATH_INFO: `/v3/projects/c4549474063b468e9dce32b91e4efa7c/users/783a44e853ed4c43b2e861c27b811375/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:56.834358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7683ec7-4d3a-4d22-ab18-4ef0f4cd3267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4549474063b468e9dce32b91e4efa7c, user_id=783a44e853ed4c43b2e861c27b811375, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:56.838511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7683ec7-4d3a-4d22-ab18-4ef0f4cd3267 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:56.949097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1070/2133] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:53:56 2022] PUT /identity/v3/projects/c4549474063b468e9dce32b91e4efa7c/users/783a44e853ed4c43b2e861c27b811375/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:53:56.961065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d1ba0-95ea-48f9-bdf7-12fd23b75df4 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:56.961370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d1ba0-95ea-48f9-bdf7-12fd23b75df4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:56.961811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d1ba0-95ea-48f9-bdf7-12fd23b75df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:56.999858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-6e9d1ba0-95ea-48f9-bdf7-12fd23b75df4 None None] MFA Rules not processed for user `783a44e853ed4c43b2e861c27b811375`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:53:57.001539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6e9d1ba0-95ea-48f9-bdf7-12fd23b75df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:57.132308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1064/2134] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:53:57.142195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7683ec7-4d3a-4d22-ab18-4ef0f4cd3267 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:53:57.144256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f7683ec7-4d3a-4d22-ab18-4ef0f4cd3267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:53:57.258129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:53:57.281597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7683ec7-4d3a-4d22-ab18-4ef0f4cd3267 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:53:57.282739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751b3e3-fd7c-4484-8596-75ce09956b9e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33003a9bf4454f5bac2e6ad73544efc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:53:57.283363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751b3e3-fd7c-4484-8596-75ce09956b9e service glance] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:53:57.283640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751b3e3-fd7c-4484-8596-75ce09956b9e service glance] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:53:57.283911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751b3e3-fd7c-4484-8596-75ce09956b9e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:53:57.297050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751b3e3-fd7c-4484-8596-75ce09956b9e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:53:57.309459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751b3e3-fd7c-4484-8596-75ce09956b9e service glance] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:53:57.330710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1071/2135] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:53:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:54:07.114697 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:07.353023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:32.748892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d1ba0-95ea-48f9-bdf7-12fd23b75df4 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:32.813494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d1ba0-95ea-48f9-bdf7-12fd23b75df4 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:32.817909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:32.818549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:32.818823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:32.819092 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] PATH_INFO: `/v3/users/783a44e853ed4c43b2e861c27b811375` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:32.820207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=783a44e853ed4c43b2e861c27b811375)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:32.822139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:33.016807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '783a44e853ed4c43b2e861c27b811375'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:33.104749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '783a44e853ed4c43b2e861c27b811375'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:33.112000 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '783a44e853ed4c43b2e861c27b811375'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:33.124779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '783a44e853ed4c43b2e861c27b811375'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:33.125531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '783a44e853ed4c43b2e861c27b811375'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:33.131497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1065/2136] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:54:32 2022] DELETE /identity/v3/users/783a44e853ed4c43b2e861c27b811375 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:54:33.136812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302cbca-22cc-43ab-93f5-3c9f2d30dfc1 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:33.137577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302cbca-22cc-43ab-93f5-3c9f2d30dfc1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:33.137577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302cbca-22cc-43ab-93f5-3c9f2d30dfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:33.156691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2302cbca-22cc-43ab-93f5-3c9f2d30dfc1 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:33.157884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2302cbca-22cc-43ab-93f5-3c9f2d30dfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:33.217890 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1072/2137] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:54:33.227515 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:33.228932 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:33.287653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2c6f5-d33e-47f6-aaed-935e32d974c9 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:33.292477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d7217-f4b8-4955-a2df-29b7cbdf9471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:33.293135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d7217-f4b8-4955-a2df-29b7cbdf9471 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:33.293411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d7217-f4b8-4955-a2df-29b7cbdf9471 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:33.293678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d7217-f4b8-4955-a2df-29b7cbdf9471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:33.303673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d7217-f4b8-4955-a2df-29b7cbdf9471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:33.305114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d7217-f4b8-4955-a2df-29b7cbdf9471 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:33.315763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1066/2138] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:54:33.741635 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302cbca-22cc-43ab-93f5-3c9f2d30dfc1 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:33.742021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2302cbca-22cc-43ab-93f5-3c9f2d30dfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:33.756456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302cbca-22cc-43ab-93f5-3c9f2d30dfc1 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:33.761339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:33.762031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:33.762335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:33.762631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] PATH_INFO: `/v3/projects/c4549474063b468e9dce32b91e4efa7c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:33.763846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4549474063b468e9dce32b91e4efa7c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:33.765748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:33.874015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4549474063b468e9dce32b91e4efa7c', 'request_id': 'req-5e394451-efa2-4071-9ac4-79f6470e849e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:33.953860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4549474063b468e9dce32b91e4efa7c', 'request_id': 'req-5e394451-efa2-4071-9ac4-79f6470e849e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:33.961567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4549474063b468e9dce32b91e4efa7c', 'request_id': 'req-5e394451-efa2-4071-9ac4-79f6470e849e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:33.976027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] The token cache is being invalidate because project c4549474063b468e9dce32b91e4efa7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:54:33.976324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:33.977028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5e394451-efa2-4071-9ac4-79f6470e849e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:33.978564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1073/2139] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:54:33 2022] DELETE /identity/v3/projects/c4549474063b468e9dce32b91e4efa7c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:54:33.984569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19340af-250b-47ec-891f-3399ac48011a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:33.985085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19340af-250b-47ec-891f-3399ac48011a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:33.985187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19340af-250b-47ec-891f-3399ac48011a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.004562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-f19340af-250b-47ec-891f-3399ac48011a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:34.005792 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-f19340af-250b-47ec-891f-3399ac48011a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:34.024433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:34.069183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1067/2140] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:34.077003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656549-393c-4479-bae0-2e78e7c9c408 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:34.077290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656549-393c-4479-bae0-2e78e7c9c408 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:34.077587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656549-393c-4479-bae0-2e78e7c9c408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.097744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-78656549-393c-4479-bae0-2e78e7c9c408 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:34.098968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-78656549-393c-4479-bae0-2e78e7c9c408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:34.107494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1074/2141] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:34.111675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19340af-250b-47ec-891f-3399ac48011a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:34.121266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19340af-250b-47ec-891f-3399ac48011a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:34.126039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c1c18-0c5d-49d8-94ef-ed3403a6ea7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:34.126639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c1c18-0c5d-49d8-94ef-ed3403a6ea7e admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:34.126910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c1c18-0c5d-49d8-94ef-ed3403a6ea7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:34.127176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5c1c18-0c5d-49d8-94ef-ed3403a6ea7e admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.127837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c1c18-0c5d-49d8-94ef-ed3403a6ea7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:54:34.128134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c1c18-0c5d-49d8-94ef-ed3403a6ea7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:34.129749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5c1c18-0c5d-49d8-94ef-ed3403a6ea7e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:34.142666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1068/2142] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:54:34.147862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78656549-393c-4479-bae0-2e78e7c9c408 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:34.158550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78656549-393c-4479-bae0-2e78e7c9c408 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:34.163376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b075db-b241-443d-9217-4eca513934ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:34.163998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b075db-b241-443d-9217-4eca513934ec admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:34.164271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b075db-b241-443d-9217-4eca513934ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:34.164533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b075db-b241-443d-9217-4eca513934ec admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.165158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b075db-b241-443d-9217-4eca513934ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:34.167086 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b075db-b241-443d-9217-4eca513934ec admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:34.284496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1075/2143] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:54:34 2022] POST /identity/v3/projects => generated 361 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:54:34.288430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c1c18-0c5d-49d8-94ef-ed3403a6ea7e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:34.298505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5c1c18-0c5d-49d8-94ef-ed3403a6ea7e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:34.355787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6540a-75e2-4881-8e0b-cdaa6dd28a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:34.356379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6540a-75e2-4881-8e0b-cdaa6dd28a40 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:34.356676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6540a-75e2-4881-8e0b-cdaa6dd28a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:34.356927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6540a-75e2-4881-8e0b-cdaa6dd28a40 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.357579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6540a-75e2-4881-8e0b-cdaa6dd28a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:34.359507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6540a-75e2-4881-8e0b-cdaa6dd28a40 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:34.436765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1069/2144] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:54:34 2022] POST /identity/v3/users => generated 281 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:54:34.440897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b075db-b241-443d-9217-4eca513934ec admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:34.451748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b075db-b241-443d-9217-4eca513934ec admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:34.456309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc739da-051c-48a8-bc94-2a4e4f8c7fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:34.456899 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc739da-051c-48a8-bc94-2a4e4f8c7fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:34.457174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc739da-051c-48a8-bc94-2a4e4f8c7fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:34.457475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc739da-051c-48a8-bc94-2a4e4f8c7fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.458160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc739da-051c-48a8-bc94-2a4e4f8c7fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:34.459835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc739da-051c-48a8-bc94-2a4e4f8c7fe2 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:34.470449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1076/2145] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:54:34.474638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6540a-75e2-4881-8e0b-cdaa6dd28a40 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:34.484681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6540a-75e2-4881-8e0b-cdaa6dd28a40 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:34.489190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922be58-eeeb-4c3f-ba22-82ae46e262e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:34.489764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922be58-eeeb-4c3f-ba22-82ae46e262e8 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:34.490064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922be58-eeeb-4c3f-ba22-82ae46e262e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:34.490332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922be58-eeeb-4c3f-ba22-82ae46e262e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.490957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922be58-eeeb-4c3f-ba22-82ae46e262e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:34.492597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922be58-eeeb-4c3f-ba22-82ae46e262e8 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:34.503347 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1070/2146] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:54:34.515430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc739da-051c-48a8-bc94-2a4e4f8c7fe2 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:34.525663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc739da-051c-48a8-bc94-2a4e4f8c7fe2 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:34.530627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8781f-95a4-4154-9d7b-d3d8814a76de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:34.531295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8781f-95a4-4154-9d7b-d3d8814a76de admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:34.531571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8781f-95a4-4154-9d7b-d3d8814a76de admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:34.531841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8781f-95a4-4154-9d7b-d3d8814a76de admin admin] PATH_INFO: `/v3/projects/d54924350762481c8416c50bf33a28a9/users/0ca4c49e5c9647b1aebe1738d83c7d17/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.546493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8781f-95a4-4154-9d7b-d3d8814a76de admin admin] RBAC: Authorizing `identity:create_grant(project_id=d54924350762481c8416c50bf33a28a9, user_id=0ca4c49e5c9647b1aebe1738d83c7d17, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:34.549048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8781f-95a4-4154-9d7b-d3d8814a76de admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:34.650967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1077/2147] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:54:34 2022] PUT /identity/v3/projects/d54924350762481c8416c50bf33a28a9/users/0ca4c49e5c9647b1aebe1738d83c7d17/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:54:34.661542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c86f88-5f13-4cb8-8528-d79c3150b71e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:34.661824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c86f88-5f13-4cb8-8528-d79c3150b71e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:34.662134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c86f88-5f13-4cb8-8528-d79c3150b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.690853 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-80c86f88-5f13-4cb8-8528-d79c3150b71e None None] MFA Rules not processed for user `0ca4c49e5c9647b1aebe1738d83c7d17`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:34.695628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-80c86f88-5f13-4cb8-8528-d79c3150b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:34.757974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1071/2148] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:34.793646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8781f-95a4-4154-9d7b-d3d8814a76de admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:34.795472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f4f8781f-95a4-4154-9d7b-d3d8814a76de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:34.818444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:34.853632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8781f-95a4-4154-9d7b-d3d8814a76de admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:34.858344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4aa86b-3778-4b53-98be-4f02d2cd46e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:34.858947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4aa86b-3778-4b53-98be-4f02d2cd46e5 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:34.859223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4aa86b-3778-4b53-98be-4f02d2cd46e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:34.859490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4aa86b-3778-4b53-98be-4f02d2cd46e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.867926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4aa86b-3778-4b53-98be-4f02d2cd46e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:34.869574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4aa86b-3778-4b53-98be-4f02d2cd46e5 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:34.879063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1078/2149] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:54:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:54:34.894046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c86f88-5f13-4cb8-8528-d79c3150b71e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:34.915498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:34.980735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c86f88-5f13-4cb8-8528-d79c3150b71e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:34.989647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:34.990849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:34.991324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:34.991802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] PATH_INFO: `/v3/users/0ca4c49e5c9647b1aebe1738d83c7d17` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:34.993774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca4c49e5c9647b1aebe1738d83c7d17)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:34.998002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:35.198185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca4c49e5c9647b1aebe1738d83c7d17'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:35.284836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca4c49e5c9647b1aebe1738d83c7d17'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:35.293402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca4c49e5c9647b1aebe1738d83c7d17'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:35.306283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca4c49e5c9647b1aebe1738d83c7d17'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:35.310569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca4c49e5c9647b1aebe1738d83c7d17'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:35.313246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1072/2150] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:54:34 2022] DELETE /identity/v3/users/0ca4c49e5c9647b1aebe1738d83c7d17 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:54:35.317772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566ff2f-30d8-4f80-b164-32eea3f4d247 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:35.318326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566ff2f-30d8-4f80-b164-32eea3f4d247 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:35.318326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566ff2f-30d8-4f80-b164-32eea3f4d247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:35.341277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-7566ff2f-30d8-4f80-b164-32eea3f4d247 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:35.342438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-7566ff2f-30d8-4f80-b164-32eea3f4d247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:35.402408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1079/2151] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:54:35.413651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:35.415243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:35.485173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e10df5-0c64-463d-90fa-18888b2dadbc admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:35.489009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b6a0d-dfa1-4cdf-af4f-1d65d0534801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:35.489603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6a0d-dfa1-4cdf-af4f-1d65d0534801 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:35.489880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6a0d-dfa1-4cdf-af4f-1d65d0534801 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:35.490195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6a0d-dfa1-4cdf-af4f-1d65d0534801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:35.499103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b6a0d-dfa1-4cdf-af4f-1d65d0534801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:35.500560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b6a0d-dfa1-4cdf-af4f-1d65d0534801 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:35.512651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1073/2152] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:54:35.968548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566ff2f-30d8-4f80-b164-32eea3f4d247 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:35.970349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-7566ff2f-30d8-4f80-b164-32eea3f4d247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:35.983789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566ff2f-30d8-4f80-b164-32eea3f4d247 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:35.988648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:35.989380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:35.989728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:35.990091 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] PATH_INFO: `/v3/projects/d54924350762481c8416c50bf33a28a9` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:35.991296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d54924350762481c8416c50bf33a28a9)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:35.993246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:36.111341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd54924350762481c8416c50bf33a28a9', 'request_id': 'req-fada451d-601e-4582-83a0-ebe4e9274389'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:36.164754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd54924350762481c8416c50bf33a28a9', 'request_id': 'req-fada451d-601e-4582-83a0-ebe4e9274389'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:36.172995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd54924350762481c8416c50bf33a28a9', 'request_id': 'req-fada451d-601e-4582-83a0-ebe4e9274389'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:36.184233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] The token cache is being invalidate because project d54924350762481c8416c50bf33a28a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:54:36.184603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:36.185158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-fada451d-601e-4582-83a0-ebe4e9274389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:36.188334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1080/2153] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:54:35 2022] DELETE /identity/v3/projects/d54924350762481c8416c50bf33a28a9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:54:36.193462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aded0c6-dad0-40ea-8f5a-2c9f48cc3571 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:36.196667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aded0c6-dad0-40ea-8f5a-2c9f48cc3571 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:36.196667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aded0c6-dad0-40ea-8f5a-2c9f48cc3571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:36.219441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5aded0c6-dad0-40ea-8f5a-2c9f48cc3571 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:36.220564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5aded0c6-dad0-40ea-8f5a-2c9f48cc3571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:36.266292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:36.318777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1074/2154] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:36.329597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8dd03-c0ac-4b41-9427-042d43365a6d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:36.331225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8dd03-c0ac-4b41-9427-042d43365a6d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:36.331225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8dd03-c0ac-4b41-9427-042d43365a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:36.366999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9fd8dd03-c0ac-4b41-9427-042d43365a6d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:36.368400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9fd8dd03-c0ac-4b41-9427-042d43365a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:36.380345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1081/2155] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:36.384858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aded0c6-dad0-40ea-8f5a-2c9f48cc3571 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:36.401098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aded0c6-dad0-40ea-8f5a-2c9f48cc3571 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:36.407352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e62519-2f0b-40cb-8b81-f4bfe03125e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:36.408176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e62519-2f0b-40cb-8b81-f4bfe03125e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:36.408522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e62519-2f0b-40cb-8b81-f4bfe03125e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:36.408800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e62519-2f0b-40cb-8b81-f4bfe03125e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:36.411677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e62519-2f0b-40cb-8b81-f4bfe03125e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:54:36.412374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e62519-2f0b-40cb-8b81-f4bfe03125e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:36.414500 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e62519-2f0b-40cb-8b81-f4bfe03125e3 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:36.430968 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1075/2156] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 16:54:36.436298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8dd03-c0ac-4b41-9427-042d43365a6d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:36.458179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8dd03-c0ac-4b41-9427-042d43365a6d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:36.458179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e370d-0c59-4a75-831f-2ac4937fb274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:36.458179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e370d-0c59-4a75-831f-2ac4937fb274 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:36.458179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e370d-0c59-4a75-831f-2ac4937fb274 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:36.458547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e370d-0c59-4a75-831f-2ac4937fb274 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:36.458547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e370d-0c59-4a75-831f-2ac4937fb274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:36.458547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e370d-0c59-4a75-831f-2ac4937fb274 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:36.575361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1082/2157] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:54:36 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:54:36.578784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e62519-2f0b-40cb-8b81-f4bfe03125e3 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:36.589641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e62519-2f0b-40cb-8b81-f4bfe03125e3 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:36.653652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa10a23-bdfd-4e82-af19-32d7a4d25ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:36.654303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa10a23-bdfd-4e82-af19-32d7a4d25ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:36.654576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa10a23-bdfd-4e82-af19-32d7a4d25ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:36.654847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa10a23-bdfd-4e82-af19-32d7a4d25ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:36.655477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa10a23-bdfd-4e82-af19-32d7a4d25ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:36.657424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa10a23-bdfd-4e82-af19-32d7a4d25ae7 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:36.739192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1076/2158] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:54:36 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:54:36.744142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e370d-0c59-4a75-831f-2ac4937fb274 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:36.755210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e370d-0c59-4a75-831f-2ac4937fb274 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:36.759802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153462b1-6e30-40c8-9021-d900440f4953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:36.760384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153462b1-6e30-40c8-9021-d900440f4953 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:36.760656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153462b1-6e30-40c8-9021-d900440f4953 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:36.760918 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153462b1-6e30-40c8-9021-d900440f4953 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:36.761578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153462b1-6e30-40c8-9021-d900440f4953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:36.763311 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153462b1-6e30-40c8-9021-d900440f4953 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:36.773479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1083/2159] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:54:36.778578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa10a23-bdfd-4e82-af19-32d7a4d25ae7 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:36.788704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa10a23-bdfd-4e82-af19-32d7a4d25ae7 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:36.794043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20eedc-fb13-4d62-9d92-32e0cb047712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:36.794636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20eedc-fb13-4d62-9d92-32e0cb047712 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:36.794909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20eedc-fb13-4d62-9d92-32e0cb047712 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:36.795176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20eedc-fb13-4d62-9d92-32e0cb047712 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:36.795807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20eedc-fb13-4d62-9d92-32e0cb047712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:36.797381 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20eedc-fb13-4d62-9d92-32e0cb047712 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:36.808621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1077/2160] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:54:36.812784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153462b1-6e30-40c8-9021-d900440f4953 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:36.823236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153462b1-6e30-40c8-9021-d900440f4953 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:36.827849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d4e27-7116-4113-a3af-3542edef7dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:36.828490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d4e27-7116-4113-a3af-3542edef7dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:36.828772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d4e27-7116-4113-a3af-3542edef7dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:36.829045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d4e27-7116-4113-a3af-3542edef7dd4 admin admin] PATH_INFO: `/v3/projects/44a1eac2bb574b76ac976808a5c7d17c/users/ba6e04f4392146808d342394907a6735/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:36.831264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:36.845064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d4e27-7116-4113-a3af-3542edef7dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a1eac2bb574b76ac976808a5c7d17c, user_id=ba6e04f4392146808d342394907a6735, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:36.847767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d4e27-7116-4113-a3af-3542edef7dd4 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:36.932920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1084/2161] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:54:36 2022] PUT /identity/v3/projects/44a1eac2bb574b76ac976808a5c7d17c/users/ba6e04f4392146808d342394907a6735/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:54:36.943633 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f373-ad36-4e14-aec3-60ddcc66bb09 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:36.944032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f373-ad36-4e14-aec3-60ddcc66bb09 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:36.945311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796f373-ad36-4e14-aec3-60ddcc66bb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:36.972319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-4796f373-ad36-4e14-aec3-60ddcc66bb09 None None] MFA Rules not processed for user `ba6e04f4392146808d342394907a6735`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:36.973514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4796f373-ad36-4e14-aec3-60ddcc66bb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:37.039991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1078/2162] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:37.051285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d4e27-7116-4113-a3af-3542edef7dd4 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:37.053599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d09d4e27-7116-4113-a3af-3542edef7dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:37.110038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d4e27-7116-4113-a3af-3542edef7dd4 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:37.113688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a837c-c1ad-48f8-b75e-4f7d9812f545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:37.114326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a837c-c1ad-48f8-b75e-4f7d9812f545 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:37.114604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a837c-c1ad-48f8-b75e-4f7d9812f545 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:37.114874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a837c-c1ad-48f8-b75e-4f7d9812f545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:37.123360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a837c-c1ad-48f8-b75e-4f7d9812f545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:37.124772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a837c-c1ad-48f8-b75e-4f7d9812f545 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:37.133529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1085/2163] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:54:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:54:42.290900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796f373-ad36-4e14-aec3-60ddcc66bb09 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:42.331708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:42.355649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796f373-ad36-4e14-aec3-60ddcc66bb09 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:42.361775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:42.362413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:42.362695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:42.362955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] PATH_INFO: `/v3/users/ba6e04f4392146808d342394907a6735` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:42.364074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba6e04f4392146808d342394907a6735)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:42.366186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:42.598168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba6e04f4392146808d342394907a6735'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:42.672772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba6e04f4392146808d342394907a6735'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:42.680208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba6e04f4392146808d342394907a6735'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:42.693056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba6e04f4392146808d342394907a6735'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:42.693711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba6e04f4392146808d342394907a6735'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:42.699791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1079/2164] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 16:54:42 2022] DELETE /identity/v3/users/ba6e04f4392146808d342394907a6735 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:54:42.704588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f896a-27bb-4bb8-bc63-311ed0c03e3d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:42.705094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f896a-27bb-4bb8-bc63-311ed0c03e3d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:42.705190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f896a-27bb-4bb8-bc63-311ed0c03e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:42.724216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c44f896a-27bb-4bb8-bc63-311ed0c03e3d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:42.725570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c44f896a-27bb-4bb8-bc63-311ed0c03e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:42.785378 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1086/2165] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:42.795863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:42.797259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:42.855379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9d313-532f-4b96-85c4-c6873e8a074a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:42.859034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57c6c4-a8f8-431a-ab82-6c4239f59e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:42.859626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c6c4-a8f8-431a-ab82-6c4239f59e50 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:42.859897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c6c4-a8f8-431a-ab82-6c4239f59e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:42.860160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57c6c4-a8f8-431a-ab82-6c4239f59e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:42.869650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57c6c4-a8f8-431a-ab82-6c4239f59e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:42.871299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57c6c4-a8f8-431a-ab82-6c4239f59e50 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:42.890356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1080/2166] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:54:43.281184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f896a-27bb-4bb8-bc63-311ed0c03e3d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:43.282673 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c44f896a-27bb-4bb8-bc63-311ed0c03e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:43.301368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f896a-27bb-4bb8-bc63-311ed0c03e3d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:43.307789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:43.308432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:43.308710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:43.308974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] PATH_INFO: `/v3/projects/44a1eac2bb574b76ac976808a5c7d17c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:43.310185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a1eac2bb574b76ac976808a5c7d17c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:43.312003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:43.442709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a1eac2bb574b76ac976808a5c7d17c', 'request_id': 'req-b26e0db9-c0e6-479e-be34-a15424876025'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:43.510439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a1eac2bb574b76ac976808a5c7d17c', 'request_id': 'req-b26e0db9-c0e6-479e-be34-a15424876025'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:43.519413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a1eac2bb574b76ac976808a5c7d17c', 'request_id': 'req-b26e0db9-c0e6-479e-be34-a15424876025'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:43.531362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] The token cache is being invalidate because project 44a1eac2bb574b76ac976808a5c7d17c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 16:54:43.531362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:43.532452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b26e0db9-c0e6-479e-be34-a15424876025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 16:54:43.533533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1087/2167] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 16:54:43 2022] DELETE /identity/v3/projects/44a1eac2bb574b76ac976808a5c7d17c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:54:43.538535 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52a9cd-5ff8-46da-976b-b98c3da46f52 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:43.538990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52a9cd-5ff8-46da-976b-b98c3da46f52 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:43.539082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52a9cd-5ff8-46da-976b-b98c3da46f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:43.560640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ca52a9cd-5ff8-46da-976b-b98c3da46f52 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:43.561758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ca52a9cd-5ff8-46da-976b-b98c3da46f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:43.613424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:43.623421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1081/2168] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:43.632243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30316a6-507e-437c-be23-7ebb106c2fe7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:43.632518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30316a6-507e-437c-be23-7ebb106c2fe7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:43.632781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30316a6-507e-437c-be23-7ebb106c2fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:43.653669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b30316a6-507e-437c-be23-7ebb106c2fe7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:43.654934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b30316a6-507e-437c-be23-7ebb106c2fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:43.666027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1088/2169] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:43.668797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52a9cd-5ff8-46da-976b-b98c3da46f52 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:43.679287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52a9cd-5ff8-46da-976b-b98c3da46f52 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:43.684509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e9d21-831e-45a2-8e65-dc56b4952d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:43.685285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e9d21-831e-45a2-8e65-dc56b4952d25 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:43.685560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e9d21-831e-45a2-8e65-dc56b4952d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:43.685827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e9d21-831e-45a2-8e65-dc56b4952d25 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:43.686505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e9d21-831e-45a2-8e65-dc56b4952d25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 16:54:43.686812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e9d21-831e-45a2-8e65-dc56b4952d25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:43.688432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e9d21-831e-45a2-8e65-dc56b4952d25 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:43.701756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1082/2170] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 16:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 16:54:43.706243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30316a6-507e-437c-be23-7ebb106c2fe7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:43.714078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:43.723422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30316a6-507e-437c-be23-7ebb106c2fe7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:43.729506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838f8e9-c426-4be5-ae32-0c5b8169be6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:43.730150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838f8e9-c426-4be5-ae32-0c5b8169be6f admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:43.730430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838f8e9-c426-4be5-ae32-0c5b8169be6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:43.730842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838f8e9-c426-4be5-ae32-0c5b8169be6f admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:43.731640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838f8e9-c426-4be5-ae32-0c5b8169be6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:43.734004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838f8e9-c426-4be5-ae32-0c5b8169be6f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:43.855822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1089/2171] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 16:54:43 2022] POST /identity/v3/projects => generated 363 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 16:54:43.859974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e9d21-831e-45a2-8e65-dc56b4952d25 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:43.870663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e9d21-831e-45a2-8e65-dc56b4952d25 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:43.929109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e859d-6a41-4c77-b23f-2b361702132a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:43.929747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e859d-6a41-4c77-b23f-2b361702132a admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:43.930044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e859d-6a41-4c77-b23f-2b361702132a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:43.930313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e859d-6a41-4c77-b23f-2b361702132a admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:43.930940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e859d-6a41-4c77-b23f-2b361702132a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:43.933508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e859d-6a41-4c77-b23f-2b361702132a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:44.051394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1083/2172] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 16:54:43 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 16:54:44.052592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838f8e9-c426-4be5-ae32-0c5b8169be6f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:44.064277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838f8e9-c426-4be5-ae32-0c5b8169be6f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:44.069002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dd891-dec8-448c-9c0b-60b111171324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:44.069598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dd891-dec8-448c-9c0b-60b111171324 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:44.069874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dd891-dec8-448c-9c0b-60b111171324 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:44.070182 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27dd891-dec8-448c-9c0b-60b111171324 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:44.070839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dd891-dec8-448c-9c0b-60b111171324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:44.072433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27dd891-dec8-448c-9c0b-60b111171324 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:44.082312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1090/2173] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:54:44.087155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e859d-6a41-4c77-b23f-2b361702132a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:44.097196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e859d-6a41-4c77-b23f-2b361702132a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:44.101870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a2fbb-bac3-4952-8642-54ab12bef4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:44.102504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2fbb-bac3-4952-8642-54ab12bef4ae admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:44.102782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2fbb-bac3-4952-8642-54ab12bef4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:44.103054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a2fbb-bac3-4952-8642-54ab12bef4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:44.104251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a2fbb-bac3-4952-8642-54ab12bef4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:44.105883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a2fbb-bac3-4952-8642-54ab12bef4ae admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:44.116195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1084/2174] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 16:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 16:54:44.122332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dd891-dec8-448c-9c0b-60b111171324 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:44.132659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27dd891-dec8-448c-9c0b-60b111171324 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:44.137543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160dd92-72ef-4448-bd53-94a53ab60e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:44.138291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160dd92-72ef-4448-bd53-94a53ab60e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:44.138571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160dd92-72ef-4448-bd53-94a53ab60e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:44.138838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160dd92-72ef-4448-bd53-94a53ab60e11 admin admin] PATH_INFO: `/v3/projects/04330028a6ab4d6fa51f3767f9bbed1c/users/0550ce5a53054042b80c90a2a539b1c9/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:44.154948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160dd92-72ef-4448-bd53-94a53ab60e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04330028a6ab4d6fa51f3767f9bbed1c, user_id=0550ce5a53054042b80c90a2a539b1c9, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:44.157790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160dd92-72ef-4448-bd53-94a53ab60e11 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:44.240773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1091/2175] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 16:54:44 2022] PUT /identity/v3/projects/04330028a6ab4d6fa51f3767f9bbed1c/users/0550ce5a53054042b80c90a2a539b1c9/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 16:54:44.251832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a4a7e-5e19-49cb-9dd8-7b8c4c564640 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:44.252122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a4a7e-5e19-49cb-9dd8-7b8c4c564640 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:44.252423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a4a7e-5e19-49cb-9dd8-7b8c4c564640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:44.280614 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-110a4a7e-5e19-49cb-9dd8-7b8c4c564640 None None] MFA Rules not processed for user `0550ce5a53054042b80c90a2a539b1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:44.281841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-110a4a7e-5e19-49cb-9dd8-7b8c4c564640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:44.349673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1085/2176] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:44.360804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160dd92-72ef-4448-bd53-94a53ab60e11 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:44.362605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-1160dd92-72ef-4448-bd53-94a53ab60e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:44.421474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160dd92-72ef-4448-bd53-94a53ab60e11 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:44.425270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb4837-9245-4efc-b0f5-6bc3e5686d9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:44.425826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb4837-9245-4efc-b0f5-6bc3e5686d9f service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:44.426121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb4837-9245-4efc-b0f5-6bc3e5686d9f service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:44.426391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb4837-9245-4efc-b0f5-6bc3e5686d9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:44.437443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb4837-9245-4efc-b0f5-6bc3e5686d9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:44.438127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb4837-9245-4efc-b0f5-6bc3e5686d9f service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:44.448291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1092/2177] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:54:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:54:46.573128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aabe0-e071-43bd-b4d7-b00742cd2a48 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:46.573579 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aabe0-e071-43bd-b4d7-b00742cd2a48 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:46.573669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aabe0-e071-43bd-b4d7-b00742cd2a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:46.599399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8d7aabe0-e071-43bd-b4d7-b00742cd2a48 None None] MFA Rules not processed for user `be94daa21a15438f8ff005e44b133363`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:54:46.599399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8d7aabe0-e071-43bd-b4d7-b00742cd2a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:46.654055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:54:46.668225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1086/2178] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:54:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:54:48.846799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb4837-9245-4efc-b0f5-6bc3e5686d9f service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:48.848211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d7cb4837-9245-4efc-b0f5-6bc3e5686d9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:48.920246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb4837-9245-4efc-b0f5-6bc3e5686d9f service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:48.925180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b514b-0229-4940-80fb-c34abba06c13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:48.925764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b514b-0229-4940-80fb-c34abba06c13 service placement] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:48.926069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b514b-0229-4940-80fb-c34abba06c13 service placement] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:48.926334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b514b-0229-4940-80fb-c34abba06c13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:48.928019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b65b514b-0229-4940-80fb-c34abba06c13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:48.939849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b514b-0229-4940-80fb-c34abba06c13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:48.941399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b514b-0229-4940-80fb-c34abba06c13 service placement] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:48.952127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1093/2179] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:54:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:54:55.228645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7aabe0-e071-43bd-b4d7-b00742cd2a48 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:54:55.232846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8d7aabe0-e071-43bd-b4d7-b00742cd2a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:55.290277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7aabe0-e071-43bd-b4d7-b00742cd2a48 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:54:55.293788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661ccc4-5cb4-48c6-969d-db7e99b89d7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:54:55.294415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661ccc4-5cb4-48c6-969d-db7e99b89d7a service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:54:55.294685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661ccc4-5cb4-48c6-969d-db7e99b89d7a service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:54:55.294949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661ccc4-5cb4-48c6-969d-db7e99b89d7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:54:55.296642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-f661ccc4-5cb4-48c6-969d-db7e99b89d7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:54:55.357712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661ccc4-5cb4-48c6-969d-db7e99b89d7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:54:55.359164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661ccc4-5cb4-48c6-969d-db7e99b89d7a service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:54:55.372098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1087/2180] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:54:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:54:58.947060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=81,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:55:05.367496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=51 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:55:43.498454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93ec24-23af-455a-aa5c-4a53c041fb0a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:55:43.503966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93ec24-23af-455a-aa5c-4a53c041fb0a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:55:43.503966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93ec24-23af-455a-aa5c-4a53c041fb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:55:43.518791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9b93ec24-23af-455a-aa5c-4a53c041fb0a None None] MFA Rules not processed for user `6b7457ab864e4df6b10a195f8500c14c`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:55:43.519994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9b93ec24-23af-455a-aa5c-4a53c041fb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:55:43.577339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1094/2181] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 16:55:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:55:53.570610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=25 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:56:36.810204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661ccc4-5cb4-48c6-969d-db7e99b89d7a service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:56:36.819135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661ccc4-5cb4-48c6-969d-db7e99b89d7a service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:56:36.824247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b25d64-940b-479f-8a67-9850b89f1267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:56:36.824850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b25d64-940b-479f-8a67-9850b89f1267 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:56:36.825127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b25d64-940b-479f-8a67-9850b89f1267 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:56:36.825399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b25d64-940b-479f-8a67-9850b89f1267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:56:36.827156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-32b25d64-940b-479f-8a67-9850b89f1267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:56:36.897265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b25d64-940b-479f-8a67-9850b89f1267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:56:36.898730 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b25d64-940b-479f-8a67-9850b89f1267 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:56:36.910182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1088/2182] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:56:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:56:36.986547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93ec24-23af-455a-aa5c-4a53c041fb0a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:56:36.996407 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93ec24-23af-455a-aa5c-4a53c041fb0a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:56:37.000100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f206d-6847-4c6e-9c78-7ddb77e3a152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:56:37.000686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f206d-6847-4c6e-9c78-7ddb77e3a152 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:56:37.000958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f206d-6847-4c6e-9c78-7ddb77e3a152 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:56:37.001220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f206d-6847-4c6e-9c78-7ddb77e3a152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:56:37.002978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-891f206d-6847-4c6e-9c78-7ddb77e3a152 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:56:37.017259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f206d-6847-4c6e-9c78-7ddb77e3a152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:56:37.018703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f206d-6847-4c6e-9c78-7ddb77e3a152 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:56:37.031108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1095/2183] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:56:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:56:46.905285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=34 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:56:47.025738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=6 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:56:48.157400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b25d64-940b-479f-8a67-9850b89f1267 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:56:48.159067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-32b25d64-940b-479f-8a67-9850b89f1267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:56:48.217723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b25d64-940b-479f-8a67-9850b89f1267 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:56:48.221574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aadd2-c5f7-43d1-8277-76735286577d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33003a9bf4454f5bac2e6ad73544efc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:56:48.222211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aadd2-c5f7-43d1-8277-76735286577d service glance] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:56:48.222478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aadd2-c5f7-43d1-8277-76735286577d service glance] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:56:48.222749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aadd2-c5f7-43d1-8277-76735286577d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:56:48.224481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c89aadd2-c5f7-43d1-8277-76735286577d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:56:48.283277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89aadd2-c5f7-43d1-8277-76735286577d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:56:48.284702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89aadd2-c5f7-43d1-8277-76735286577d service glance] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:56:48.294423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1089/2184] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:56:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:56:48.763277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb6e17-80a6-44e7-864b-523a00937e01 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:56:48.763779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb6e17-80a6-44e7-864b-523a00937e01 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:56:48.763883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb6e17-80a6-44e7-864b-523a00937e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:56:48.783817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f4fb6e17-80a6-44e7-864b-523a00937e01 None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:56:48.785095 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f4fb6e17-80a6-44e7-864b-523a00937e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:56:48.841159 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1096/2185] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 16:56:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:56:48.886041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aadd2-c5f7-43d1-8277-76735286577d service glance] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:56:48.887427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c89aadd2-c5f7-43d1-8277-76735286577d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:56:48.958237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aadd2-c5f7-43d1-8277-76735286577d service glance] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:56:48.964014 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c0a29-3fa2-42c1-823c-17a6a901f173 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:56:48.964866 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c0a29-3fa2-42c1-823c-17a6a901f173 service swift] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:56:48.965357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c0a29-3fa2-42c1-823c-17a6a901f173 service swift] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:56:48.965653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c0a29-3fa2-42c1-823c-17a6a901f173 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:56:48.974099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c0a29-3fa2-42c1-823c-17a6a901f173 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:56:48.976000 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c0a29-3fa2-42c1-823c-17a6a901f173 service swift] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:56:48.984806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1090/2186] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 16:56:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 16:56:54.411386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb6e17-80a6-44e7-864b-523a00937e01 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:56:54.413216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f4fb6e17-80a6-44e7-864b-523a00937e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:56:54.431618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb6e17-80a6-44e7-864b-523a00937e01 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:56:54.437974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff12c3-a2f0-4f51-91ff-ebc830286dc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:56:54.438944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff12c3-a2f0-4f51-91ff-ebc830286dc7 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:56:54.439169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff12c3-a2f0-4f51-91ff-ebc830286dc7 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:56:54.439439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff12c3-a2f0-4f51-91ff-ebc830286dc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:56:54.441189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-42ff12c3-a2f0-4f51-91ff-ebc830286dc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:56:54.511518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff12c3-a2f0-4f51-91ff-ebc830286dc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:56:54.513238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff12c3-a2f0-4f51-91ff-ebc830286dc7 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:56:54.523943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1097/2187] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:56:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:56:58.981721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=80 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:57:04.519001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=53 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:57:07.790731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c0a29-3fa2-42c1-823c-17a6a901f173 service swift] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:57:07.807693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c0a29-3fa2-42c1-823c-17a6a901f173 service swift] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:57:07.813797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855b21e-4327-4cbd-8491-1689703b04b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:57:07.814807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855b21e-4327-4cbd-8491-1689703b04b2 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:07.815263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855b21e-4327-4cbd-8491-1689703b04b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:07.815695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855b21e-4327-4cbd-8491-1689703b04b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:07.818495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5855b21e-4327-4cbd-8491-1689703b04b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:57:07.835460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855b21e-4327-4cbd-8491-1689703b04b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:57:07.837839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855b21e-4327-4cbd-8491-1689703b04b2 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:57:07.851567 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1091/2188] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:57:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:57:08.314249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4daf4-0184-4280-b55b-12395036240a None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:08.315167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4daf4-0184-4280-b55b-12395036240a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:08.315598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4daf4-0184-4280-b55b-12395036240a None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:08.319135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1098/2189] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 16:57:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:57:08.326563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a015b-7568-448e-b6eb-1ec78142714a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:08.326563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a015b-7568-448e-b6eb-1ec78142714a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:08.327647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a015b-7568-448e-b6eb-1ec78142714a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:08.350156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d40a015b-7568-448e-b6eb-1ec78142714a None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:57:08.351656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d40a015b-7568-448e-b6eb-1ec78142714a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:57:08.363126 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1092/2190] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 16:57:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:57:08.375280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4daf4-0184-4280-b55b-12395036240a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:57:08.394186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4daf4-0184-4280-b55b-12395036240a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:57:08.403172 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed0e0a-3b42-40df-8109-f5c1bc5138c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33003a9bf4454f5bac2e6ad73544efc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:57:08.403793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed0e0a-3b42-40df-8109-f5c1bc5138c9 service glance] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:08.404062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed0e0a-3b42-40df-8109-f5c1bc5138c9 service glance] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:08.404528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed0e0a-3b42-40df-8109-f5c1bc5138c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:08.416536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed0e0a-3b42-40df-8109-f5c1bc5138c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:57:08.417983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed0e0a-3b42-40df-8109-f5c1bc5138c9 service glance] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:57:08.428995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1099/2191] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:57:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:57:08.697658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c9ca-16f7-4145-a6fd-5e14cad34955 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:08.697864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c9ca-16f7-4145-a6fd-5e14cad34955 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:08.698142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2c9ca-16f7-4145-a6fd-5e14cad34955 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:08.699496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1093/2192] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 16:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:57:08.704639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb610ff-3023-4161-ba67-e770602fb61a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:08.704961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb610ff-3023-4161-ba67-e770602fb61a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:08.705272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb610ff-3023-4161-ba67-e770602fb61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:08.724966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-beb610ff-3023-4161-ba67-e770602fb61a None None] MFA Rules not processed for user `c2f11dcf2c3c41a0b435a90af74c2e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:57:08.726208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-beb610ff-3023-4161-ba67-e770602fb61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:57:08.732806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1100/2193] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 16:57:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 16:57:08.746742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2c9ca-16f7-4145-a6fd-5e14cad34955 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:57:08.757563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2c9ca-16f7-4145-a6fd-5e14cad34955 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:57:08.761621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583fd3e-7758-4cb9-a850-2ca84631d9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:57:08.762232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583fd3e-7758-4cb9-a850-2ca84631d9ab service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:08.762503 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583fd3e-7758-4cb9-a850-2ca84631d9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:08.762767 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583fd3e-7758-4cb9-a850-2ca84631d9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:08.770969 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583fd3e-7758-4cb9-a850-2ca84631d9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:57:08.771480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583fd3e-7758-4cb9-a850-2ca84631d9ab service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:57:08.779529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1094/2194] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:57:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:57:08.862777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7516a-7e83-4b09-b914-c4ab0e11cefc None None] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:08.863123 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7516a-7e83-4b09-b914-c4ab0e11cefc None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:08.863368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7516a-7e83-4b09-b914-c4ab0e11cefc None None] PATH_INFO: `/` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:08.864775 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1101/2195] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 16:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 16:57:08.870757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdfbe1-460a-4b07-84eb-ecef24e15e5e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:08.871230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdfbe1-460a-4b07-84eb-ecef24e15e5e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:08.871325 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cdfbe1-460a-4b07-84eb-ecef24e15e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:08.890068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c1cdfbe1-460a-4b07-84eb-ecef24e15e5e None None] MFA Rules not processed for user `5d17aa4b23e04290b6c8b1924b0dcf32`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 16:57:08.891249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c1cdfbe1-460a-4b07-84eb-ecef24e15e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:57:08.900649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1095/2196] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 16:57:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 16:57:08.911976 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7516a-7e83-4b09-b914-c4ab0e11cefc None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:57:08.923721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7516a-7e83-4b09-b914-c4ab0e11cefc None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:57:08.927804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d2cee-fd10-4d52-8824-458f087fe42e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:57:08.928534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2cee-fd10-4d52-8824-458f087fe42e service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:08.928877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2cee-fd10-4d52-8824-458f087fe42e service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:08.929213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d2cee-fd10-4d52-8824-458f087fe42e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:08.940966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d2cee-fd10-4d52-8824-458f087fe42e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:57:08.942498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d2cee-fd10-4d52-8824-458f087fe42e service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:57:08.953806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1102/2197] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:57:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:57:09.021281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdfbe1-460a-4b07-84eb-ecef24e15e5e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:57:09.033326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cdfbe1-460a-4b07-84eb-ecef24e15e5e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:57:09.038559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0b4ab-c097-4825-8598-2f35ea675ff9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:57:09.039150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0b4ab-c097-4825-8598-2f35ea675ff9 service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:09.039451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0b4ab-c097-4825-8598-2f35ea675ff9 service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:09.039690 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf0b4ab-c097-4825-8598-2f35ea675ff9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:09.041458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7bf0b4ab-c097-4825-8598-2f35ea675ff9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:57:09.056429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0b4ab-c097-4825-8598-2f35ea675ff9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:57:09.058820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf0b4ab-c097-4825-8598-2f35ea675ff9 service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:57:09.068624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1096/2198] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:57:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:57:18.948747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=15 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:57:19.064427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=24 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:57:31.315773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d2cee-fd10-4d52-8824-458f087fe42e service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:57:31.317278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ab2d2cee-fd10-4d52-8824-458f087fe42e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:57:31.331658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d2cee-fd10-4d52-8824-458f087fe42e service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:57:31.337112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4131d-50c9-4430-8044-37bc8f6e6b65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:57:31.337696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4131d-50c9-4430-8044-37bc8f6e6b65 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:31.338067 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4131d-50c9-4430-8044-37bc8f6e6b65 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:31.338341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4131d-50c9-4430-8044-37bc8f6e6b65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:31.340017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b9c4131d-50c9-4430-8044-37bc8f6e6b65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:57:31.350206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4131d-50c9-4430-8044-37bc8f6e6b65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:57:31.351760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4131d-50c9-4430-8044-37bc8f6e6b65 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:57:31.361019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1103/2199] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:57:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:57:32.827122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0b4ab-c097-4825-8598-2f35ea675ff9 service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:57:32.836887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf0b4ab-c097-4825-8598-2f35ea675ff9 service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:57:32.840624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ead817-6a01-4bf0-b4dc-0119699ef42a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:57:32.841233 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ead817-6a01-4bf0-b4dc-0119699ef42a service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:57:32.841507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ead817-6a01-4bf0-b4dc-0119699ef42a service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:57:32.841773 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ead817-6a01-4bf0-b4dc-0119699ef42a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:57:32.843507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b1ead817-6a01-4bf0-b4dc-0119699ef42a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:57:32.852982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ead817-6a01-4bf0-b4dc-0119699ef42a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:57:32.854929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ead817-6a01-4bf0-b4dc-0119699ef42a service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:57:32.862725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1097/2200] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:57:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:57:41.356980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=6 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:57:42.859223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=6 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:58:31.723417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4131d-50c9-4430-8044-37bc8f6e6b65 service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:58:31.736194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4131d-50c9-4430-8044-37bc8f6e6b65 service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:58:31.743080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e213f7-6a8b-4df0-ae8e-98c93ae63f7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:58:31.743689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e213f7-6a8b-4df0-ae8e-98c93ae63f7a service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:58:31.743967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e213f7-6a8b-4df0-ae8e-98c93ae63f7a service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:58:31.744238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e213f7-6a8b-4df0-ae8e-98c93ae63f7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:58:31.747030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-67e213f7-6a8b-4df0-ae8e-98c93ae63f7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:58:31.765609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e213f7-6a8b-4df0-ae8e-98c93ae63f7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:58:31.768005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e213f7-6a8b-4df0-ae8e-98c93ae63f7a service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:58:31.779297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1104/2201] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:58:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:58:35.508336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ead817-6a01-4bf0-b4dc-0119699ef42a service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:58:35.528326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ead817-6a01-4bf0-b4dc-0119699ef42a service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:58:35.534612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affeb64-546b-41df-9bf8-1d5b67307f27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:58:35.535214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affeb64-546b-41df-9bf8-1d5b67307f27 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:58:35.535490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affeb64-546b-41df-9bf8-1d5b67307f27 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:58:35.535938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affeb64-546b-41df-9bf8-1d5b67307f27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:58:35.540864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9affeb64-546b-41df-9bf8-1d5b67307f27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:58:35.576669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affeb64-546b-41df-9bf8-1d5b67307f27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:58:35.579568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affeb64-546b-41df-9bf8-1d5b67307f27 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:58:35.598523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1098/2202] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:58:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:58:41.774542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=6 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:58:45.589780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=6 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:58:50.589785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e213f7-6a8b-4df0-ae8e-98c93ae63f7a service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:58:50.606755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e213f7-6a8b-4df0-ae8e-98c93ae63f7a service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:58:50.610472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262a7c8-797c-4c95-88ee-e091e8dc94bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:58:50.611076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262a7c8-797c-4c95-88ee-e091e8dc94bb service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:58:50.611359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262a7c8-797c-4c95-88ee-e091e8dc94bb service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:58:50.611622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262a7c8-797c-4c95-88ee-e091e8dc94bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:58:50.613479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d262a7c8-797c-4c95-88ee-e091e8dc94bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:58:50.694807 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262a7c8-797c-4c95-88ee-e091e8dc94bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:58:50.696444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262a7c8-797c-4c95-88ee-e091e8dc94bb service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:58:50.705456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1105/2203] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:58:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 16:59:00.701547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=28 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:59:03.908648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affeb64-546b-41df-9bf8-1d5b67307f27 service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:59:03.920223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affeb64-546b-41df-9bf8-1d5b67307f27 service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:59:03.925779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66aaf1-a3a0-4b1c-8706-9128be2058f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:59:03.926418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66aaf1-a3a0-4b1c-8706-9128be2058f7 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:59:03.926696 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66aaf1-a3a0-4b1c-8706-9128be2058f7 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:59:03.926966 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66aaf1-a3a0-4b1c-8706-9128be2058f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:59:03.928791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4f66aaf1-a3a0-4b1c-8706-9128be2058f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:59:03.988673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66aaf1-a3a0-4b1c-8706-9128be2058f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:59:03.990434 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66aaf1-a3a0-4b1c-8706-9128be2058f7 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:59:04.000922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1099/2204] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:59:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:59:06.196350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262a7c8-797c-4c95-88ee-e091e8dc94bb service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:59:06.216950 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262a7c8-797c-4c95-88ee-e091e8dc94bb service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:59:06.223553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3517e-d97b-4ce8-95a0-80d9f831356d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:59:06.224882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3517e-d97b-4ce8-95a0-80d9f831356d service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:59:06.225173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3517e-d97b-4ce8-95a0-80d9f831356d service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:59:06.225445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3517e-d97b-4ce8-95a0-80d9f831356d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:59:06.232911 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6fd3517e-d97b-4ce8-95a0-80d9f831356d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 16:59:06.248021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3517e-d97b-4ce8-95a0-80d9f831356d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:59:06.252128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3517e-d97b-4ce8-95a0-80d9f831356d service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:59:06.273311 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1106/2205] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:59:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:59:13.996403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=28 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:59:16.267184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=6 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:59:43.422676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66aaf1-a3a0-4b1c-8706-9128be2058f7 service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:59:43.448626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66aaf1-a3a0-4b1c-8706-9128be2058f7 service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:59:43.452825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfb812-9b58-457c-9d45-0bdf6ee17ae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:59:43.453413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfb812-9b58-457c-9d45-0bdf6ee17ae8 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:59:43.453689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfb812-9b58-457c-9d45-0bdf6ee17ae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:59:43.453968 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbfb812-9b58-457c-9d45-0bdf6ee17ae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:59:43.478970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbfb812-9b58-457c-9d45-0bdf6ee17ae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:59:43.480408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbfb812-9b58-457c-9d45-0bdf6ee17ae8 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:59:43.489814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1100/2206] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:59:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:59:49.020786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3517e-d97b-4ce8-95a0-80d9f831356d service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 16:59:49.102277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3517e-d97b-4ce8-95a0-80d9f831356d service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 16:59:49.107892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cbe3e-b58f-4e3a-bc05-28e8576f8e1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 16:59:49.108497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cbe3e-b58f-4e3a-bc05-28e8576f8e1e service placement] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 16:59:49.108760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cbe3e-b58f-4e3a-bc05-28e8576f8e1e service placement] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 16:59:49.109035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4cbe3e-b58f-4e3a-bc05-28e8576f8e1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 16:59:49.126777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4cbe3e-b58f-4e3a-bc05-28e8576f8e1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 16:59:49.128905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4cbe3e-b58f-4e3a-bc05-28e8576f8e1e service placement] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 16:59:49.145432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1107/2207] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 16:59:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 16:59:53.485644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=17 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 16:59:59.137689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=27 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:01:37.183846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfb812-9b58-457c-9d45-0bdf6ee17ae8 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:01:37.204510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbfb812-9b58-457c-9d45-0bdf6ee17ae8 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:01:37.208236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44052b18-4a92-40f0-994c-9396510600fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:01:37.208848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44052b18-4a92-40f0-994c-9396510600fe service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:01:37.209125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44052b18-4a92-40f0-994c-9396510600fe service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:01:37.209393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44052b18-4a92-40f0-994c-9396510600fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:01:37.228185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44052b18-4a92-40f0-994c-9396510600fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:01:37.229672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44052b18-4a92-40f0-994c-9396510600fe service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:01:37.252495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1101/2208] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:01:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:01:38.451092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cbe3e-b58f-4e3a-bc05-28e8576f8e1e service placement] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:01:38.461399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4cbe3e-b58f-4e3a-bc05-28e8576f8e1e service placement] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:01:38.465237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5b5f2-3aad-461d-839c-fcec3ee792a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:01:38.465834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5b5f2-3aad-461d-839c-fcec3ee792a5 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:01:38.466135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5b5f2-3aad-461d-839c-fcec3ee792a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:01:38.466401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5b5f2-3aad-461d-839c-fcec3ee792a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:01:38.478346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5b5f2-3aad-461d-839c-fcec3ee792a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:01:38.479988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5b5f2-3aad-461d-839c-fcec3ee792a5 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:01:38.493780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1108/2209] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:01:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:01:47.249214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=17 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:01:48.487321 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=6 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:01:49.180942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44052b18-4a92-40f0-994c-9396510600fe service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:01:49.194042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44052b18-4a92-40f0-994c-9396510600fe service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:01:49.202052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9185209-5928-4527-83cd-c042276466aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:01:49.203061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185209-5928-4527-83cd-c042276466aa service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:01:49.203437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185209-5928-4527-83cd-c042276466aa service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:01:49.203760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185209-5928-4527-83cd-c042276466aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:01:49.217306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9185209-5928-4527-83cd-c042276466aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:01:49.217306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9185209-5928-4527-83cd-c042276466aa service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:01:49.257018 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1102/2210] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:01:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:01:55.539268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5b5f2-3aad-461d-839c-fcec3ee792a5 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:01:55.540714 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-49b5b5f2-3aad-461d-839c-fcec3ee792a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:01:55.587903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5b5f2-3aad-461d-839c-fcec3ee792a5 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:01:55.591626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893acf8-1bd5-4d8d-9c92-105bb31083ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:01:55.592224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893acf8-1bd5-4d8d-9c92-105bb31083ca service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:01:55.592497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893acf8-1bd5-4d8d-9c92-105bb31083ca service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:01:55.592760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893acf8-1bd5-4d8d-9c92-105bb31083ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:01:55.602471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893acf8-1bd5-4d8d-9c92-105bb31083ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:01:55.604037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893acf8-1bd5-4d8d-9c92-105bb31083ca service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:01:55.613561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1109/2211] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:01:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:01:59.254225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=13 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:02:05.609130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=19 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:02:31.802229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9185209-5928-4527-83cd-c042276466aa service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:02:31.812736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9185209-5928-4527-83cd-c042276466aa service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:02:31.816931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fddee3-966f-4fb6-a87b-ee773d0d456a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:02:31.817532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fddee3-966f-4fb6-a87b-ee773d0d456a service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:02:31.817806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fddee3-966f-4fb6-a87b-ee773d0d456a service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:02:31.818113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fddee3-966f-4fb6-a87b-ee773d0d456a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:02:31.828076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fddee3-966f-4fb6-a87b-ee773d0d456a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:02:31.829554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fddee3-966f-4fb6-a87b-ee773d0d456a service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:02:31.860426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1103/2212] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:02:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:02:41.857478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=13 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:02:44.655417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893acf8-1bd5-4d8d-9c92-105bb31083ca service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:02:44.666109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893acf8-1bd5-4d8d-9c92-105bb31083ca service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:02:44.669772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4d660-59e8-49d1-b92d-3f2c723674bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:02:44.670397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4d660-59e8-49d1-b92d-3f2c723674bf service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:02:44.670672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4d660-59e8-49d1-b92d-3f2c723674bf service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:02:44.670967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e4d660-59e8-49d1-b92d-3f2c723674bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:02:44.679368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4d660-59e8-49d1-b92d-3f2c723674bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:02:44.680784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e4d660-59e8-49d1-b92d-3f2c723674bf service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:02:44.688817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1110/2213] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:02:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:02:54.685397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=6 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:03:09.340386 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cda532-ec75-458d-b79b-fd81e85997f9 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:09.340892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cda532-ec75-458d-b79b-fd81e85997f9 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:09.340998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cda532-ec75-458d-b79b-fd81e85997f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:09.402394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-14cda532-ec75-458d-b79b-fd81e85997f9 None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:03:09.403640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-14cda532-ec75-458d-b79b-fd81e85997f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:03:09.420833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1104/2214] 10.222.0.79 () {58 vars in 967 bytes} [Tue Aug 9 17:03:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:03:09.629843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4d660-59e8-49d1-b92d-3f2c723674bf service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:03:09.654298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e4d660-59e8-49d1-b92d-3f2c723674bf service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:03:09.659865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83636d-80f0-4a11-9ee4-351bff6ffc9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:03:09.660656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83636d-80f0-4a11-9ee4-351bff6ffc9f service swift] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:09.660996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83636d-80f0-4a11-9ee4-351bff6ffc9f service swift] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:09.661333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83636d-80f0-4a11-9ee4-351bff6ffc9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:09.670373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83636d-80f0-4a11-9ee4-351bff6ffc9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:03:09.671928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83636d-80f0-4a11-9ee4-351bff6ffc9f service swift] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:03:09.680340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1111/2215] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 17:03:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:03:12.386899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc27818-4c09-4d54-95db-dba7cec91381 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:12.387491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc27818-4c09-4d54-95db-dba7cec91381 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:12.387491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc27818-4c09-4d54-95db-dba7cec91381 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:12.388740 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1105/2216] 10.222.0.79 () {54 vars in 912 bytes} [Tue Aug 9 17:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:03:12.394901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaf871-a7cd-41a4-9366-669f144382d1 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:12.395445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaf871-a7cd-41a4-9366-669f144382d1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:12.395647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becaf871-a7cd-41a4-9366-669f144382d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:12.468015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-becaf871-a7cd-41a4-9366-669f144382d1 None None] MFA Rules not processed for user `d7bc938254514471801c88761525e883`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:03:12.469182 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-becaf871-a7cd-41a4-9366-669f144382d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:03:12.478658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1112/2217] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 17:03:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:03:12.502850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc27818-4c09-4d54-95db-dba7cec91381 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:03:12.504554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ebc27818-4c09-4d54-95db-dba7cec91381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:03:12.529891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc27818-4c09-4d54-95db-dba7cec91381 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:03:12.536867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa1498-cd94-47ab-9846-5b60405cd94e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:03:12.537127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa1498-cd94-47ab-9846-5b60405cd94e service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:12.537420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa1498-cd94-47ab-9846-5b60405cd94e service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:12.537734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa1498-cd94-47ab-9846-5b60405cd94e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:12.547789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa1498-cd94-47ab-9846-5b60405cd94e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:03:12.550923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa1498-cd94-47ab-9846-5b60405cd94e service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:03:12.562175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1106/2218] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:03:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:03:17.210874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed6f9-7a29-4ce5-a63f-6fbfbc9d6e3d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:17.211414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed6f9-7a29-4ce5-a63f-6fbfbc9d6e3d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:17.211414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ed6f9-7a29-4ce5-a63f-6fbfbc9d6e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:17.262244 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-957ed6f9-7a29-4ce5-a63f-6fbfbc9d6e3d None None] MFA Rules not processed for user `84c46c4817184c9d8554c956bb7e48dc`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:03:17.263512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-957ed6f9-7a29-4ce5-a63f-6fbfbc9d6e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:03:17.270897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1113/2219] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:03:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:03:17.609871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa1498-cd94-47ab-9846-5b60405cd94e service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:03:17.621150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa1498-cd94-47ab-9846-5b60405cd94e service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:03:17.625109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fac72b-c315-44a8-9ce2-8d110c2464a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:03:17.625708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fac72b-c315-44a8-9ce2-8d110c2464a5 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:17.626003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fac72b-c315-44a8-9ce2-8d110c2464a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:17.626271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fac72b-c315-44a8-9ce2-8d110c2464a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:17.628142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-82fac72b-c315-44a8-9ce2-8d110c2464a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:03:17.656253 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fac72b-c315-44a8-9ce2-8d110c2464a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:03:17.658385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fac72b-c315-44a8-9ce2-8d110c2464a5 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:03:17.667448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1107/2220] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:03:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:03:18.159913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed6f9-7a29-4ce5-a63f-6fbfbc9d6e3d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:03:18.174322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ed6f9-7a29-4ce5-a63f-6fbfbc9d6e3d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:03:18.190756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c5f12-02d0-4424-87bd-287b275cb996 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:03:18.191384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c5f12-02d0-4424-87bd-287b275cb996 service placement] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:18.191660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c5f12-02d0-4424-87bd-287b275cb996 service placement] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:18.191924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c5f12-02d0-4424-87bd-287b275cb996 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:18.214782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c5f12-02d0-4424-87bd-287b275cb996 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:03:18.216058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c5f12-02d0-4424-87bd-287b275cb996 service placement] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:03:18.228139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1114/2221] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:03:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:03:27.663808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=39 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:03:28.223585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=45 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:03:30.216205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fac72b-c315-44a8-9ce2-8d110c2464a5 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:03:30.230296 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fac72b-c315-44a8-9ce2-8d110c2464a5 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:03:30.235838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff0cf7-f36c-44a1-bfc1-d1a1f6f0b203 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:03:30.236427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff0cf7-f36c-44a1-bfc1-d1a1f6f0b203 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:30.236704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff0cf7-f36c-44a1-bfc1-d1a1f6f0b203 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:30.236973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff0cf7-f36c-44a1-bfc1-d1a1f6f0b203 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:30.238742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0bff0cf7-f36c-44a1-bfc1-d1a1f6f0b203 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:03:30.268804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff0cf7-f36c-44a1-bfc1-d1a1f6f0b203 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:03:30.268804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bff0cf7-f36c-44a1-bfc1-d1a1f6f0b203 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:03:30.280479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1108/2222] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:03:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:03:31.702104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c5f12-02d0-4424-87bd-287b275cb996 service placement] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:03:31.715585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c5f12-02d0-4424-87bd-287b275cb996 service placement] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:03:31.721592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527474cf-591f-44e4-a20f-205053da47a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:03:31.722296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527474cf-591f-44e4-a20f-205053da47a9 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:31.722577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527474cf-591f-44e4-a20f-205053da47a9 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:31.722848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527474cf-591f-44e4-a20f-205053da47a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:31.732350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527474cf-591f-44e4-a20f-205053da47a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:03:31.733970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527474cf-591f-44e4-a20f-205053da47a9 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:03:31.762115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1115/2223] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:03:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:03:35.904192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff0cf7-f36c-44a1-bfc1-d1a1f6f0b203 service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:03:35.923241 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff0cf7-f36c-44a1-bfc1-d1a1f6f0b203 service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:03:35.930598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b64e0-caf8-4ac9-bfaa-5aca4c933aff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:03:35.931509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b64e0-caf8-4ac9-bfaa-5aca4c933aff service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:35.932112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b64e0-caf8-4ac9-bfaa-5aca4c933aff service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:35.932606 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b64e0-caf8-4ac9-bfaa-5aca4c933aff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:35.960392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b64e0-caf8-4ac9-bfaa-5aca4c933aff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:03:35.962069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b64e0-caf8-4ac9-bfaa-5aca4c933aff service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:03:35.987695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1109/2224] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:03:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:03:41.759332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=13 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:03:45.968962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=17 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:03:51.151692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527474cf-591f-44e4-a20f-205053da47a9 service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:03:51.164547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527474cf-591f-44e4-a20f-205053da47a9 service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:03:51.170632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b8588-cfe7-4ee3-8934-383ab7edd714 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:03:51.171398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b8588-cfe7-4ee3-8934-383ab7edd714 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:03:51.171758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b8588-cfe7-4ee3-8934-383ab7edd714 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:03:51.172159 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b8588-cfe7-4ee3-8934-383ab7edd714 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:03:51.183066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b8588-cfe7-4ee3-8934-383ab7edd714 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:03:51.183269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b8588-cfe7-4ee3-8934-383ab7edd714 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:03:51.214452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1116/2225] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:03:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:04:01.211854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=13 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:04:03.727990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b64e0-caf8-4ac9-bfaa-5aca4c933aff service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:04:03.741932 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b64e0-caf8-4ac9-bfaa-5aca4c933aff service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:04:03.749009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06289e8c-9f44-45bd-84f8-3d08c95f8789 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:04:03.749616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06289e8c-9f44-45bd-84f8-3d08c95f8789 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:04:03.749890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06289e8c-9f44-45bd-84f8-3d08c95f8789 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:04:03.750224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06289e8c-9f44-45bd-84f8-3d08c95f8789 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:04:03.759235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06289e8c-9f44-45bd-84f8-3d08c95f8789 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:04:03.760819 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06289e8c-9f44-45bd-84f8-3d08c95f8789 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:04:03.794360 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1110/2226] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:04:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:04:13.791662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=13 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:04:43.682306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b8588-cfe7-4ee3-8934-383ab7edd714 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:04:43.691432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b8588-cfe7-4ee3-8934-383ab7edd714 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:04:43.695035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a5fd5-2892-48e7-9eb8-81c6f29eeb70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:04:43.695745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a5fd5-2892-48e7-9eb8-81c6f29eeb70 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:04:43.695882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a5fd5-2892-48e7-9eb8-81c6f29eeb70 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:04:43.696154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a5fd5-2892-48e7-9eb8-81c6f29eeb70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:04:43.705070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a5fd5-2892-48e7-9eb8-81c6f29eeb70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:04:43.706506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a5fd5-2892-48e7-9eb8-81c6f29eeb70 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:04:43.715830 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1117/2227] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:04:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:04:50.974756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06289e8c-9f44-45bd-84f8-3d08c95f8789 service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:04:50.976905 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-06289e8c-9f44-45bd-84f8-3d08c95f8789 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:04:51.051372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06289e8c-9f44-45bd-84f8-3d08c95f8789 service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:04:51.057314 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875a460-3c9e-466c-9a94-b3e7a2defef1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:04:51.057907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875a460-3c9e-466c-9a94-b3e7a2defef1 service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:04:51.058205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875a460-3c9e-466c-9a94-b3e7a2defef1 service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:04:51.058589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d875a460-3c9e-466c-9a94-b3e7a2defef1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:04:51.060478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d875a460-3c9e-466c-9a94-b3e7a2defef1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:04:51.075537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875a460-3c9e-466c-9a94-b3e7a2defef1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:04:51.077107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d875a460-3c9e-466c-9a94-b3e7a2defef1 service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:04:51.087783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1111/2228] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:04:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:04:53.711749 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=6 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:05:01.083980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=32 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:06:48.923753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a5fd5-2892-48e7-9eb8-81c6f29eeb70 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:06:48.925395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-279a5fd5-2892-48e7-9eb8-81c6f29eeb70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:06:48.983097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a5fd5-2892-48e7-9eb8-81c6f29eeb70 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:06:48.986769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0dfa1-c9c7-4b5e-8d47-0df053413b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:06:48.987380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0dfa1-c9c7-4b5e-8d47-0df053413b2c service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:06:48.987665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0dfa1-c9c7-4b5e-8d47-0df053413b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:06:48.987933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0dfa1-c9c7-4b5e-8d47-0df053413b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:06:48.989773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8df0dfa1-c9c7-4b5e-8d47-0df053413b2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:06:49.052431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0dfa1-c9c7-4b5e-8d47-0df053413b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:06:49.053897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0dfa1-c9c7-4b5e-8d47-0df053413b2c service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:06:49.063497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1118/2229] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:06:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:06:54.613628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875a460-3c9e-466c-9a94-b3e7a2defef1 service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:06:54.617122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d875a460-3c9e-466c-9a94-b3e7a2defef1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:06:54.733919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d875a460-3c9e-466c-9a94-b3e7a2defef1 service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:06:54.739703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37796b6f-017a-4697-be99-ba6883be7ead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:06:54.740819 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37796b6f-017a-4697-be99-ba6883be7ead service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:06:54.741079 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37796b6f-017a-4697-be99-ba6883be7ead service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:06:54.741345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37796b6f-017a-4697-be99-ba6883be7ead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:06:54.743263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-37796b6f-017a-4697-be99-ba6883be7ead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:06:54.837307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37796b6f-017a-4697-be99-ba6883be7ead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:06:54.839001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37796b6f-017a-4697-be99-ba6883be7ead service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:06:54.857768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1112/2230] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:06:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:06:59.059160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=48 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:07:04.855116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=69 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:07:15.911640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0dfa1-c9c7-4b5e-8d47-0df053413b2c service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:15.913304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8df0dfa1-c9c7-4b5e-8d47-0df053413b2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:16.018530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0dfa1-c9c7-4b5e-8d47-0df053413b2c service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:16.024463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:16.024727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:16.025043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:16.025317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] PATH_INFO: `/v3/users/0550ce5a53054042b80c90a2a539b1c9` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:16.039898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0550ce5a53054042b80c90a2a539b1c9)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:16.042312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:16.186430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0550ce5a53054042b80c90a2a539b1c9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:07:16.245554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0550ce5a53054042b80c90a2a539b1c9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:07:16.259217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0550ce5a53054042b80c90a2a539b1c9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:07:16.263873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0550ce5a53054042b80c90a2a539b1c9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:07:16.263873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0550ce5a53054042b80c90a2a539b1c9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:07:16.274123 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1119/2231] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:07:15 2022] DELETE /identity/v3/users/0550ce5a53054042b80c90a2a539b1c9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:07:16.278671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ccde0-62db-4859-bec5-953c83ad9cde None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:16.279188 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ccde0-62db-4859-bec5-953c83ad9cde None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:16.279308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ccde0-62db-4859-bec5-953c83ad9cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:16.309252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-304ccde0-62db-4859-bec5-953c83ad9cde None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:07:16.310510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-304ccde0-62db-4859-bec5-953c83ad9cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:16.373205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1113/2232] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:07:16.386245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:16.387861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:16.434768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:07:16.452190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703f85b-f801-4839-aa02-c66667421727 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:16.458061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0009ed4-dc03-4deb-94a2-d2e08c9a1d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:16.458685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009ed4-dc03-4deb-94a2-d2e08c9a1d74 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:16.458960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009ed4-dc03-4deb-94a2-d2e08c9a1d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:16.459224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009ed4-dc03-4deb-94a2-d2e08c9a1d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:16.469266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0009ed4-dc03-4deb-94a2-d2e08c9a1d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:16.470754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0009ed4-dc03-4deb-94a2-d2e08c9a1d74 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:16.483487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1120/2233] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:07:17.012900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ccde0-62db-4859-bec5-953c83ad9cde None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:17.014455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-304ccde0-62db-4859-bec5-953c83ad9cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:17.027464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ccde0-62db-4859-bec5-953c83ad9cde None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:17.032481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:17.033115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:17.033417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:17.034313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] PATH_INFO: `/v3/projects/04330028a6ab4d6fa51f3767f9bbed1c` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:17.049705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04330028a6ab4d6fa51f3767f9bbed1c)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:17.051712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:17.140279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04330028a6ab4d6fa51f3767f9bbed1c', 'request_id': 'req-001ba868-6e6b-468b-9e36-015efd9d526d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:07:17.187871 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04330028a6ab4d6fa51f3767f9bbed1c', 'request_id': 'req-001ba868-6e6b-468b-9e36-015efd9d526d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:07:17.195964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04330028a6ab4d6fa51f3767f9bbed1c', 'request_id': 'req-001ba868-6e6b-468b-9e36-015efd9d526d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:07:17.206907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] The token cache is being invalidate because project 04330028a6ab4d6fa51f3767f9bbed1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:07:17.207186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:07:17.207940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-001ba868-6e6b-468b-9e36-015efd9d526d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:07:17.209413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1114/2234] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:07:17 2022] DELETE /identity/v3/projects/04330028a6ab4d6fa51f3767f9bbed1c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:07:17.215609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbdbbe-1031-4f95-a28d-d333155d4286 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:17.217353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbdbbe-1031-4f95-a28d-d333155d4286 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:17.217353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbdbbe-1031-4f95-a28d-d333155d4286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:17.261709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-3ecbdbbe-1031-4f95-a28d-d333155d4286 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:07:17.262866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3ecbdbbe-1031-4f95-a28d-d333155d4286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:17.331871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1121/2235] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:07:17.340450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffba4f-876b-4b62-8492-243ee88d3c82 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:17.341001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffba4f-876b-4b62-8492-243ee88d3c82 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:17.341001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffba4f-876b-4b62-8492-243ee88d3c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:17.361753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d7ffba4f-876b-4b62-8492-243ee88d3c82 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:07:17.363066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d7ffba4f-876b-4b62-8492-243ee88d3c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:17.371736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1115/2236] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:07:17.376108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbdbbe-1031-4f95-a28d-d333155d4286 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:17.386836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbdbbe-1031-4f95-a28d-d333155d4286 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:17.393178 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2895a2-5f19-4173-948d-63adec14b177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:17.393776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2895a2-5f19-4173-948d-63adec14b177 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:17.394085 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2895a2-5f19-4173-948d-63adec14b177 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:17.394355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2895a2-5f19-4173-948d-63adec14b177 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:17.394992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2895a2-5f19-4173-948d-63adec14b177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:07:17.395291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2895a2-5f19-4173-948d-63adec14b177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:17.396863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2895a2-5f19-4173-948d-63adec14b177 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:17.410924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1122/2237] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:07:17.415346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffba4f-876b-4b62-8492-243ee88d3c82 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:17.424952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffba4f-876b-4b62-8492-243ee88d3c82 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:17.430129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070a4a2-0142-4887-a2a6-9634de7c5304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:17.430846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a4a2-0142-4887-a2a6-9634de7c5304 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:17.431103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a4a2-0142-4887-a2a6-9634de7c5304 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:17.431367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a4a2-0142-4887-a2a6-9634de7c5304 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:17.432024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070a4a2-0142-4887-a2a6-9634de7c5304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:17.433985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070a4a2-0142-4887-a2a6-9634de7c5304 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:17.573047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1116/2238] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:07:17 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:07:17.579518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2895a2-5f19-4173-948d-63adec14b177 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:17.590715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2895a2-5f19-4173-948d-63adec14b177 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:17.673786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0345c-321a-4d1b-917d-92625bba10ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:17.673786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0345c-321a-4d1b-917d-92625bba10ca admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:17.673786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0345c-321a-4d1b-917d-92625bba10ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:17.673786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0345c-321a-4d1b-917d-92625bba10ca admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:17.674245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0345c-321a-4d1b-917d-92625bba10ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:17.674245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0345c-321a-4d1b-917d-92625bba10ca admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:17.749243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1123/2239] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:07:17 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:07:17.753629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070a4a2-0142-4887-a2a6-9634de7c5304 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:17.765564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070a4a2-0142-4887-a2a6-9634de7c5304 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:17.770906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb3098-db22-4e45-b71b-dcc4c8e73502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:17.771469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb3098-db22-4e45-b71b-dcc4c8e73502 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:17.771805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb3098-db22-4e45-b71b-dcc4c8e73502 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:17.772117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb3098-db22-4e45-b71b-dcc4c8e73502 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:17.772798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb3098-db22-4e45-b71b-dcc4c8e73502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:17.774488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb3098-db22-4e45-b71b-dcc4c8e73502 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:17.785295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1117/2240] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:07:17.789515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0345c-321a-4d1b-917d-92625bba10ca admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:17.799703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0345c-321a-4d1b-917d-92625bba10ca admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:17.805066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970f710-f3e9-4a2e-a7f9-6988d5932e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:17.805803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970f710-f3e9-4a2e-a7f9-6988d5932e44 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:17.805970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970f710-f3e9-4a2e-a7f9-6988d5932e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:17.806255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970f710-f3e9-4a2e-a7f9-6988d5932e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:17.806986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970f710-f3e9-4a2e-a7f9-6988d5932e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:17.808478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e970f710-f3e9-4a2e-a7f9-6988d5932e44 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:17.823285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1124/2241] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:07:17.827621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb3098-db22-4e45-b71b-dcc4c8e73502 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:17.838066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb3098-db22-4e45-b71b-dcc4c8e73502 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:17.842729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4f7d2-cdee-438e-9e66-c81851f2e87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:17.843373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f7d2-cdee-438e-9e66-c81851f2e87d admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:17.843644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f7d2-cdee-438e-9e66-c81851f2e87d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:17.843906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4f7d2-cdee-438e-9e66-c81851f2e87d admin admin] PATH_INFO: `/v3/projects/c8e76a79667843f5a03c563adfda4e7e/users/ec50e45a656b48418db227e2169ff8b7/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:17.861006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4f7d2-cdee-438e-9e66-c81851f2e87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e76a79667843f5a03c563adfda4e7e, user_id=ec50e45a656b48418db227e2169ff8b7, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:17.863676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de4f7d2-cdee-438e-9e66-c81851f2e87d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:17.866864 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:07:17.972231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1118/2242] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:07:17 2022] PUT /identity/v3/projects/c8e76a79667843f5a03c563adfda4e7e/users/ec50e45a656b48418db227e2169ff8b7/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:07:17.986022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a42dc-a0f0-45b1-bde7-a746717e6eda None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:17.986022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a42dc-a0f0-45b1-bde7-a746717e6eda None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:17.986022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a42dc-a0f0-45b1-bde7-a746717e6eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:18.013860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-127a42dc-a0f0-45b1-bde7-a746717e6eda None None] MFA Rules not processed for user `ec50e45a656b48418db227e2169ff8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:07:18.015248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-127a42dc-a0f0-45b1-bde7-a746717e6eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:18.036705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:07:18.085663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1125/2243] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:07:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:07:18.097269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4f7d2-cdee-438e-9e66-c81851f2e87d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:18.099054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5de4f7d2-cdee-438e-9e66-c81851f2e87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:18.166417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4f7d2-cdee-438e-9e66-c81851f2e87d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:18.173763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889161bb-7bc9-4fb2-a30d-3d28a891ee3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:18.174363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889161bb-7bc9-4fb2-a30d-3d28a891ee3f service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:18.174642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889161bb-7bc9-4fb2-a30d-3d28a891ee3f service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:18.174912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889161bb-7bc9-4fb2-a30d-3d28a891ee3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:18.187911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889161bb-7bc9-4fb2-a30d-3d28a891ee3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:18.189428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889161bb-7bc9-4fb2-a30d-3d28a891ee3f service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:18.198882 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1119/2244] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:07:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:07:28.090985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=17 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:07:28.194320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=28 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:07:32.023127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a42dc-a0f0-45b1-bde7-a746717e6eda None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:32.036438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a42dc-a0f0-45b1-bde7-a746717e6eda None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:32.040012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5763e78-78ff-4e4e-a410-75559fcd9c03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:32.041357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763e78-78ff-4e4e-a410-75559fcd9c03 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:32.041657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763e78-78ff-4e4e-a410-75559fcd9c03 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:32.041927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5763e78-78ff-4e4e-a410-75559fcd9c03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:32.043863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d5763e78-78ff-4e4e-a410-75559fcd9c03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:32.121133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5763e78-78ff-4e4e-a410-75559fcd9c03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:32.122607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5763e78-78ff-4e4e-a410-75559fcd9c03 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:32.131977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1126/2245] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:07:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:07:42.127670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=33 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:07:46.621922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889161bb-7bc9-4fb2-a30d-3d28a891ee3f service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:46.623446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-889161bb-7bc9-4fb2-a30d-3d28a891ee3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:46.697619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889161bb-7bc9-4fb2-a30d-3d28a891ee3f service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:46.701701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368b058-c5e6-4bb8-ac60-b112fbcc3115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:46.702358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368b058-c5e6-4bb8-ac60-b112fbcc3115 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:46.702636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368b058-c5e6-4bb8-ac60-b112fbcc3115 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:46.702914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368b058-c5e6-4bb8-ac60-b112fbcc3115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:46.704642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3368b058-c5e6-4bb8-ac60-b112fbcc3115 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:46.717448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368b058-c5e6-4bb8-ac60-b112fbcc3115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:46.719081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368b058-c5e6-4bb8-ac60-b112fbcc3115 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:46.728196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1120/2246] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:07:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:07:51.503386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5763e78-78ff-4e4e-a410-75559fcd9c03 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:07:51.513691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5763e78-78ff-4e4e-a410-75559fcd9c03 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:07:51.517732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b92a0c-68b4-4c39-94ea-32846ef03dd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:07:51.518397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b92a0c-68b4-4c39-94ea-32846ef03dd0 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:07:51.519464 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b92a0c-68b4-4c39-94ea-32846ef03dd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:07:51.519804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b92a0c-68b4-4c39-94ea-32846ef03dd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:07:51.521665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-81b92a0c-68b4-4c39-94ea-32846ef03dd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:07:51.601292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b92a0c-68b4-4c39-94ea-32846ef03dd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:07:51.602877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b92a0c-68b4-4c39-94ea-32846ef03dd0 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:07:51.615382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1127/2247] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:07:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:07:56.723975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=26 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:08:01.610338 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=34 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:08:26.141485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368b058-c5e6-4bb8-ac60-b112fbcc3115 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:08:26.151128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368b058-c5e6-4bb8-ac60-b112fbcc3115 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:08:26.155400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cd37d-2deb-4542-9713-4feeb6fb50b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:08:26.156073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cd37d-2deb-4542-9713-4feeb6fb50b5 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:08:26.156345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cd37d-2deb-4542-9713-4feeb6fb50b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:08:26.156610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cd37d-2deb-4542-9713-4feeb6fb50b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:08:26.158377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d10cd37d-2deb-4542-9713-4feeb6fb50b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:08:26.173616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cd37d-2deb-4542-9713-4feeb6fb50b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:08:26.175106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cd37d-2deb-4542-9713-4feeb6fb50b5 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:08:26.190088 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1121/2248] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:08:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:08:32.304157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b92a0c-68b4-4c39-94ea-32846ef03dd0 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:08:32.313505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b92a0c-68b4-4c39-94ea-32846ef03dd0 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:08:32.317052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5ad38-3f82-4350-84fb-eb84a1bd884b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:08:32.317676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5ad38-3f82-4350-84fb-eb84a1bd884b service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:08:32.317941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5ad38-3f82-4350-84fb-eb84a1bd884b service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:08:32.318238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5ad38-3f82-4350-84fb-eb84a1bd884b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:08:32.320060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4ae5ad38-3f82-4350-84fb-eb84a1bd884b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:08:32.393310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5ad38-3f82-4350-84fb-eb84a1bd884b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:08:32.394836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5ad38-3f82-4350-84fb-eb84a1bd884b service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:08:32.405010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1128/2249] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:08:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:08:36.181979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=6 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:08:42.400488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=33 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:08:50.868423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cd37d-2deb-4542-9713-4feeb6fb50b5 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:08:50.869932 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d10cd37d-2deb-4542-9713-4feeb6fb50b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:08:50.883866 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cd37d-2deb-4542-9713-4feeb6fb50b5 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:08:50.889339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d2c63-970a-4aa9-8c7e-7735b54ba855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:08:50.889921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d2c63-970a-4aa9-8c7e-7735b54ba855 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:08:50.890230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d2c63-970a-4aa9-8c7e-7735b54ba855 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:08:50.890502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d2c63-970a-4aa9-8c7e-7735b54ba855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:08:50.892176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c50d2c63-970a-4aa9-8c7e-7735b54ba855 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:08:50.961818 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d2c63-970a-4aa9-8c7e-7735b54ba855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:08:50.963651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d2c63-970a-4aa9-8c7e-7735b54ba855 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:08:50.973405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1122/2250] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:08:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:09:00.968945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=33 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:09:03.922471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5ad38-3f82-4350-84fb-eb84a1bd884b service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:09:03.931307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5ad38-3f82-4350-84fb-eb84a1bd884b service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:09:03.935728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e67bb2-08a0-4a08-a8a3-a99d3c0716f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:09:03.936324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e67bb2-08a0-4a08-a8a3-a99d3c0716f6 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:09:03.936601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e67bb2-08a0-4a08-a8a3-a99d3c0716f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:09:03.936870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e67bb2-08a0-4a08-a8a3-a99d3c0716f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:09:03.938764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d3e67bb2-08a0-4a08-a8a3-a99d3c0716f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:09:04.019879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e67bb2-08a0-4a08-a8a3-a99d3c0716f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:09:04.021300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e67bb2-08a0-4a08-a8a3-a99d3c0716f6 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:09:04.031207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1129/2251] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:09:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:09:14.026360 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=33 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:10:00.016504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d2c63-970a-4aa9-8c7e-7735b54ba855 service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:10:00.018710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c50d2c63-970a-4aa9-8c7e-7735b54ba855 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:10:00.184075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d2c63-970a-4aa9-8c7e-7735b54ba855 service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:10:00.191856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b6519-5cd5-44f9-bb08-5d96fca7fab0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:10:00.192485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b6519-5cd5-44f9-bb08-5d96fca7fab0 service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:10:00.192770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b6519-5cd5-44f9-bb08-5d96fca7fab0 service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:10:00.193045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b6519-5cd5-44f9-bb08-5d96fca7fab0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:10:00.195067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-710b6519-5cd5-44f9-bb08-5d96fca7fab0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:10:00.209517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b6519-5cd5-44f9-bb08-5d96fca7fab0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:10:00.211198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b6519-5cd5-44f9-bb08-5d96fca7fab0 service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:10:00.222077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1123/2252] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:10:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:10:10.218176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=54 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:10:54.357725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e67bb2-08a0-4a08-a8a3-a99d3c0716f6 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:10:54.360170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d3e67bb2-08a0-4a08-a8a3-a99d3c0716f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:10:54.441264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e67bb2-08a0-4a08-a8a3-a99d3c0716f6 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:10:54.451481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4aaec-9bb0-4d2f-ac8d-764382cd05e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:10:54.452439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4aaec-9bb0-4d2f-ac8d-764382cd05e9 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:10:54.452870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4aaec-9bb0-4d2f-ac8d-764382cd05e9 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:10:54.453333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4aaec-9bb0-4d2f-ac8d-764382cd05e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:10:54.455374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-dda4aaec-9bb0-4d2f-ac8d-764382cd05e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:10:54.479860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4aaec-9bb0-4d2f-ac8d-764382cd05e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:10:54.481693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4aaec-9bb0-4d2f-ac8d-764382cd05e9 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:10:54.515329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1130/2253] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:10:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:11:04.494947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=17 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:11:48.751218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b6519-5cd5-44f9-bb08-5d96fca7fab0 service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:11:48.760402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b6519-5cd5-44f9-bb08-5d96fca7fab0 service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:11:48.764042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c481c-7c4b-4398-9011-88db8ccd7b3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:11:48.764621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c481c-7c4b-4398-9011-88db8ccd7b3c service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:11:48.764893 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c481c-7c4b-4398-9011-88db8ccd7b3c service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:11:48.765179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c481c-7c4b-4398-9011-88db8ccd7b3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:11:48.766867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-359c481c-7c4b-4398-9011-88db8ccd7b3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:11:48.826588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c481c-7c4b-4398-9011-88db8ccd7b3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:11:48.828071 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c481c-7c4b-4398-9011-88db8ccd7b3c service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:11:48.836925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1124/2254] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:11:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:11:54.866719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4aaec-9bb0-4d2f-ac8d-764382cd05e9 service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:11:54.876899 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4aaec-9bb0-4d2f-ac8d-764382cd05e9 service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:11:54.880507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666bb4b-17ce-4af9-aebb-d71d5bac9b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:11:54.881090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666bb4b-17ce-4af9-aebb-d71d5bac9b4d service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:11:54.881364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666bb4b-17ce-4af9-aebb-d71d5bac9b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:11:54.881637 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666bb4b-17ce-4af9-aebb-d71d5bac9b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:11:54.883377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d666bb4b-17ce-4af9-aebb-d71d5bac9b4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:11:54.948238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666bb4b-17ce-4af9-aebb-d71d5bac9b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:11:54.949665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666bb4b-17ce-4af9-aebb-d71d5bac9b4d service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:11:54.959089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1131/2255] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:11:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:11:58.832741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=28 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:12:04.954611 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=28 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:12:19.370931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c481c-7c4b-4398-9011-88db8ccd7b3c service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:12:19.390068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c481c-7c4b-4398-9011-88db8ccd7b3c service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:12:19.393866 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5291e-7c0e-4d3a-a216-456a08467d5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:12:19.394480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5291e-7c0e-4d3a-a216-456a08467d5e service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:12:19.394755 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5291e-7c0e-4d3a-a216-456a08467d5e service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:12:19.395089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5291e-7c0e-4d3a-a216-456a08467d5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:12:19.403366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5291e-7c0e-4d3a-a216-456a08467d5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:12:19.404784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5291e-7c0e-4d3a-a216-456a08467d5e service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:12:19.413465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1125/2256] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:12:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:12:29.409580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=10 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:12:32.183134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666bb4b-17ce-4af9-aebb-d71d5bac9b4d service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:12:32.208279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666bb4b-17ce-4af9-aebb-d71d5bac9b4d service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:12:32.213225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd389d5c-f4bf-4d7c-a773-5b8d7a989c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:12:32.213979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd389d5c-f4bf-4d7c-a773-5b8d7a989c0c service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:12:32.214382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd389d5c-f4bf-4d7c-a773-5b8d7a989c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:12:32.214755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd389d5c-f4bf-4d7c-a773-5b8d7a989c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:12:32.225029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd389d5c-f4bf-4d7c-a773-5b8d7a989c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:12:32.226511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd389d5c-f4bf-4d7c-a773-5b8d7a989c0c service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:12:32.259438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1132/2257] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:12:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:12:42.255727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=17 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:12:51.602207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5291e-7c0e-4d3a-a216-456a08467d5e service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:12:51.622429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5291e-7c0e-4d3a-a216-456a08467d5e service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:12:51.626703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8dd1af-e62e-46c4-9cc7-328bfffaaa3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:12:51.627420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dd1af-e62e-46c4-9cc7-328bfffaaa3e service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:12:51.627698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dd1af-e62e-46c4-9cc7-328bfffaaa3e service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:12:51.627965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dd1af-e62e-46c4-9cc7-328bfffaaa3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:12:51.636233 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8dd1af-e62e-46c4-9cc7-328bfffaaa3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:12:51.637678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8dd1af-e62e-46c4-9cc7-328bfffaaa3e service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:12:51.645475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1126/2258] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:12:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:12:52.026645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd389d5c-f4bf-4d7c-a773-5b8d7a989c0c service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:12:52.037567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd389d5c-f4bf-4d7c-a773-5b8d7a989c0c service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:12:52.041915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723ab49-8ee4-4248-947e-d1281aeb41b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:12:52.045727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723ab49-8ee4-4248-947e-d1281aeb41b8 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:12:52.045727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723ab49-8ee4-4248-947e-d1281aeb41b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:12:52.045727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723ab49-8ee4-4248-947e-d1281aeb41b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:12:52.062651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723ab49-8ee4-4248-947e-d1281aeb41b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:12:52.064129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723ab49-8ee4-4248-947e-d1281aeb41b8 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:12:52.088725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1133/2259] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:12:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:13:01.642026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=10 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:13:02.084874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=13 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:13:19.938457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8dd1af-e62e-46c4-9cc7-328bfffaaa3e service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:13:20.020133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8dd1af-e62e-46c4-9cc7-328bfffaaa3e service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:13:20.025566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:13:20.026213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:13:20.026490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:13:20.026759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] PATH_INFO: `/v3/users/ec50e45a656b48418db227e2169ff8b7` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:13:20.027911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec50e45a656b48418db227e2169ff8b7)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:13:20.029825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:13:20.234448 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec50e45a656b48418db227e2169ff8b7'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:13:20.312164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec50e45a656b48418db227e2169ff8b7'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:13:20.321862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec50e45a656b48418db227e2169ff8b7'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:13:20.334859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec50e45a656b48418db227e2169ff8b7'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:13:20.335574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec50e45a656b48418db227e2169ff8b7'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:13:20.341703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1127/2260] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:13:19 2022] DELETE /identity/v3/users/ec50e45a656b48418db227e2169ff8b7 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:13:20.346250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f775b-861e-469e-9808-094a0cb0f9c7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:13:20.346751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f775b-861e-469e-9808-094a0cb0f9c7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:13:20.346888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f775b-861e-469e-9808-094a0cb0f9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:13:20.375566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-0b7f775b-861e-469e-9808-094a0cb0f9c7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:13:20.376769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0b7f775b-861e-469e-9808-094a0cb0f9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:13:20.441266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1134/2261] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:13:20.453795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:13:20.458904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:13:20.526108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcfe47-8d2f-4cb1-a9a6-477ebb9970f4 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:13:20.529909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5939a4-d860-41fc-9f11-28c41242a930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:13:20.530535 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5939a4-d860-41fc-9f11-28c41242a930 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:13:20.530809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5939a4-d860-41fc-9f11-28c41242a930 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:13:20.531075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5939a4-d860-41fc-9f11-28c41242a930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:13:20.541086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5939a4-d860-41fc-9f11-28c41242a930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:13:20.542544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5939a4-d860-41fc-9f11-28c41242a930 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:13:20.552344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1128/2262] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:13:20.986806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f775b-861e-469e-9808-094a0cb0f9c7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:13:20.991649 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0b7f775b-861e-469e-9808-094a0cb0f9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:13:21.004454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f775b-861e-469e-9808-094a0cb0f9c7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:13:21.010888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:13:21.011613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:13:21.012233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:13:21.012679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] PATH_INFO: `/v3/projects/c8e76a79667843f5a03c563adfda4e7e` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:13:21.014068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e76a79667843f5a03c563adfda4e7e)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:13:21.016312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:13:21.133079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e76a79667843f5a03c563adfda4e7e', 'request_id': 'req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:13:21.238064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e76a79667843f5a03c563adfda4e7e', 'request_id': 'req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:13:21.246121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e76a79667843f5a03c563adfda4e7e', 'request_id': 'req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:13:21.257837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] The token cache is being invalidate because project c8e76a79667843f5a03c563adfda4e7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:13:21.262710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:13:21.262710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:13:21.262710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1135/2263] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:13:20 2022] DELETE /identity/v3/projects/c8e76a79667843f5a03c563adfda4e7e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:13:25.945809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5939a4-d860-41fc-9f11-28c41242a930 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:13:25.947399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6f5939a4-d860-41fc-9f11-28c41242a930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:13:25.959381 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5939a4-d860-41fc-9f11-28c41242a930 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:13:25.967805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:13:26.011564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29b03-f281-4266-aa6c-d846ed911327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:13:26.012430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29b03-f281-4266-aa6c-d846ed911327 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:13:26.012701 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29b03-f281-4266-aa6c-d846ed911327 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:13:26.013041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed29b03-f281-4266-aa6c-d846ed911327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:13:26.014832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7ed29b03-f281-4266-aa6c-d846ed911327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:13:26.097908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed29b03-f281-4266-aa6c-d846ed911327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:13:26.099725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed29b03-f281-4266-aa6c-d846ed911327 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:13:26.112278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1129/2264] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:13:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:13:31.254221 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:13:32.285711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:13:32.287216 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:13:32.317025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbaf26-c109-4bd7-95bd-e47fea1fdc19 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:13:32.322980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e987db9-abcc-4fcc-a6e4-63a74fdd880c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:13:32.323585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e987db9-abcc-4fcc-a6e4-63a74fdd880c service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:13:32.323859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e987db9-abcc-4fcc-a6e4-63a74fdd880c service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:13:32.324125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e987db9-abcc-4fcc-a6e4-63a74fdd880c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:13:32.325890 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5e987db9-abcc-4fcc-a6e4-63a74fdd880c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:13:32.404029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e987db9-abcc-4fcc-a6e4-63a74fdd880c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:13:32.405674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e987db9-abcc-4fcc-a6e4-63a74fdd880c service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:13:32.425498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1136/2265] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:13:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:13:36.106468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=49 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:13:42.422785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=41 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:13:51.084801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29b03-f281-4266-aa6c-d846ed911327 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:13:51.086284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7ed29b03-f281-4266-aa6c-d846ed911327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:13:51.113741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed29b03-f281-4266-aa6c-d846ed911327 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:13:51.119545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacccb7-e1ca-4e33-8e18-87282cb86d31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:13:51.120138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacccb7-e1ca-4e33-8e18-87282cb86d31 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:13:51.120411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacccb7-e1ca-4e33-8e18-87282cb86d31 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:13:51.120673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacccb7-e1ca-4e33-8e18-87282cb86d31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:13:51.122518 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6dacccb7-e1ca-4e33-8e18-87282cb86d31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:13:51.179973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacccb7-e1ca-4e33-8e18-87282cb86d31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:13:51.181573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacccb7-e1ca-4e33-8e18-87282cb86d31 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:13:51.190798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1130/2266] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:13:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:14:01.186293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=33 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:14:03.791752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e987db9-abcc-4fcc-a6e4-63a74fdd880c service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:14:03.793242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5e987db9-abcc-4fcc-a6e4-63a74fdd880c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:14:03.806796 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e987db9-abcc-4fcc-a6e4-63a74fdd880c service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:14:03.810594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d40d3-1cdf-4650-894f-e81cbd029480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:14:03.811186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d40d3-1cdf-4650-894f-e81cbd029480 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:14:03.811479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d40d3-1cdf-4650-894f-e81cbd029480 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:14:03.811727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d40d3-1cdf-4650-894f-e81cbd029480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:14:03.813587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a54d40d3-1cdf-4650-894f-e81cbd029480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:14:03.883782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d40d3-1cdf-4650-894f-e81cbd029480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:14:03.885201 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d40d3-1cdf-4650-894f-e81cbd029480 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:14:03.903672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1137/2267] 10.222.0.79 () {60 vars in 1396 bytes} [Tue Aug 9 17:14:03 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:14:13.901021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=36 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:15:06.946076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacccb7-e1ca-4e33-8e18-87282cb86d31 service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:06.947492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6dacccb7-e1ca-4e33-8e18-87282cb86d31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:07.032458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacccb7-e1ca-4e33-8e18-87282cb86d31 service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:07.037626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092d1cf-960a-42a7-a2ef-d83476f13434 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:07.038266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092d1cf-960a-42a7-a2ef-d83476f13434 service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:07.038545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092d1cf-960a-42a7-a2ef-d83476f13434 service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:07.038812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092d1cf-960a-42a7-a2ef-d83476f13434 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:07.040540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d092d1cf-960a-42a7-a2ef-d83476f13434 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:07.052658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092d1cf-960a-42a7-a2ef-d83476f13434 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:07.054318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092d1cf-960a-42a7-a2ef-d83476f13434 service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:07.064555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1131/2268] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:15:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:15:17.060385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=37 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:15:47.455380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d40d3-1cdf-4650-894f-e81cbd029480 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:47.464027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d40d3-1cdf-4650-894f-e81cbd029480 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:47.467666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c443e-71ea-467d-943e-800db46a101b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:47.468256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c443e-71ea-467d-943e-800db46a101b service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:47.468530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c443e-71ea-467d-943e-800db46a101b service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:47.468798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c443e-71ea-467d-943e-800db46a101b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:47.470540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-218c443e-71ea-467d-943e-800db46a101b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:47.533384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c443e-71ea-467d-943e-800db46a101b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:47.535027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c443e-71ea-467d-943e-800db46a101b service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:47.545481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1138/2269] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:15:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:15:51.649423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092d1cf-960a-42a7-a2ef-d83476f13434 service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:51.650964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d092d1cf-960a-42a7-a2ef-d83476f13434 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:51.664520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092d1cf-960a-42a7-a2ef-d83476f13434 service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:51.670564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30bb4a-0976-4cee-a5b9-274206be9c23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:51.671098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30bb4a-0976-4cee-a5b9-274206be9c23 service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:51.671373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30bb4a-0976-4cee-a5b9-274206be9c23 service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:51.671644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30bb4a-0976-4cee-a5b9-274206be9c23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:51.673890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7d30bb4a-0976-4cee-a5b9-274206be9c23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:51.686220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30bb4a-0976-4cee-a5b9-274206be9c23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:51.687799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30bb4a-0976-4cee-a5b9-274206be9c23 service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:51.698220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1132/2270] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:15:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:15:51.796498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c443e-71ea-467d-943e-800db46a101b service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:51.797914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-218c443e-71ea-467d-943e-800db46a101b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:51.810950 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c443e-71ea-467d-943e-800db46a101b service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:51.816271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:51.816856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:51.817134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:51.817406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] PATH_INFO: `/v3/users/be94daa21a15438f8ff005e44b133363` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:51.818585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be94daa21a15438f8ff005e44b133363)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:51.820474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:52.004601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be94daa21a15438f8ff005e44b133363'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:52.086858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be94daa21a15438f8ff005e44b133363'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:52.107582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be94daa21a15438f8ff005e44b133363'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:52.108503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be94daa21a15438f8ff005e44b133363'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:52.109236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-d76c11ab-713b-44c5-8ce0-64f9bdc81d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be94daa21a15438f8ff005e44b133363'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:52.124953 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1139/2271] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:15:51 2022] DELETE /identity/v3/users/be94daa21a15438f8ff005e44b133363 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:15:52.393288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30bb4a-0976-4cee-a5b9-274206be9c23 service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:52.405934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30bb4a-0976-4cee-a5b9-274206be9c23 service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:52.462439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:52.462969 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:52.463242 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:52.463509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] PATH_INFO: `/v3/projects/01ca6df74bca45fc9edde426ba6ee069` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:52.465121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ca6df74bca45fc9edde426ba6ee069)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:52.467036 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:52.569820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ca6df74bca45fc9edde426ba6ee069', 'request_id': 'req-17f0e695-d058-4844-97e4-a8b0fa2aa89c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:52.651553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ca6df74bca45fc9edde426ba6ee069', 'request_id': 'req-17f0e695-d058-4844-97e4-a8b0fa2aa89c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:52.658622 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ca6df74bca45fc9edde426ba6ee069', 'request_id': 'req-17f0e695-d058-4844-97e4-a8b0fa2aa89c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:52.671385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] The token cache is being invalidate because project 01ca6df74bca45fc9edde426ba6ee069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:15:52.671672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:52.672355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-17f0e695-d058-4844-97e4-a8b0fa2aa89c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:52.674166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1133/2272] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:15:52 2022] DELETE /identity/v3/projects/01ca6df74bca45fc9edde426ba6ee069 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:15:52.680066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb159826-7994-47f3-bc0c-ec4aa896c9be None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:52.680345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb159826-7994-47f3-bc0c-ec4aa896c9be None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:52.680613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb159826-7994-47f3-bc0c-ec4aa896c9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:52.714464 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-cb159826-7994-47f3-bc0c-ec4aa896c9be None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:15:52.715574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-cb159826-7994-47f3-bc0c-ec4aa896c9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:52.769312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:15:52.779373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1140/2273] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:15:52.787836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9da4b-94d0-40bd-ad92-61d4f70775e4 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:52.788111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9da4b-94d0-40bd-ad92-61d4f70775e4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:52.788382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9da4b-94d0-40bd-ad92-61d4f70775e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:52.808852 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c6c9da4b-94d0-40bd-ad92-61d4f70775e4 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:15:52.810175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c6c9da4b-94d0-40bd-ad92-61d4f70775e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:52.819376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1134/2274] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:15:52.823746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb159826-7994-47f3-bc0c-ec4aa896c9be None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:52.837212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb159826-7994-47f3-bc0c-ec4aa896c9be None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:52.840572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19913761-1b0e-45fe-832e-1fc10692d27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:52.841489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19913761-1b0e-45fe-832e-1fc10692d27c admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:52.841610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19913761-1b0e-45fe-832e-1fc10692d27c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:52.842170 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19913761-1b0e-45fe-832e-1fc10692d27c admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:52.842881 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19913761-1b0e-45fe-832e-1fc10692d27c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:15:52.843178 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19913761-1b0e-45fe-832e-1fc10692d27c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:52.844798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19913761-1b0e-45fe-832e-1fc10692d27c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:52.858432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1141/2275] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:15:52.863665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9da4b-94d0-40bd-ad92-61d4f70775e4 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:52.873878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9da4b-94d0-40bd-ad92-61d4f70775e4 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:52.878745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca6326-add7-4594-8e22-cb3d06ad1426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:52.879356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca6326-add7-4594-8e22-cb3d06ad1426 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:52.879632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca6326-add7-4594-8e22-cb3d06ad1426 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:52.879900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca6326-add7-4594-8e22-cb3d06ad1426 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:52.880529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca6326-add7-4594-8e22-cb3d06ad1426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:52.882468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca6326-add7-4594-8e22-cb3d06ad1426 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:52.989074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1135/2276] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:15:52 2022] POST /identity/v3/projects => generated 405 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:15:52.993696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19913761-1b0e-45fe-832e-1fc10692d27c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:53.006507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19913761-1b0e-45fe-832e-1fc10692d27c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:53.065268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88685d01-bf87-4d21-b659-1d8db883c872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:53.065868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88685d01-bf87-4d21-b659-1d8db883c872 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:53.066174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88685d01-bf87-4d21-b659-1d8db883c872 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:53.066424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88685d01-bf87-4d21-b659-1d8db883c872 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:53.067037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88685d01-bf87-4d21-b659-1d8db883c872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:53.068927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88685d01-bf87-4d21-b659-1d8db883c872 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:53.195040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1142/2277] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:15:52 2022] POST /identity/v3/users => generated 303 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:15:53.199435 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca6326-add7-4594-8e22-cb3d06ad1426 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:53.210419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca6326-add7-4594-8e22-cb3d06ad1426 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:53.217586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e98b48-e479-4d17-aa21-8827e3db046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:53.218259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98b48-e479-4d17-aa21-8827e3db046c admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:53.218538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98b48-e479-4d17-aa21-8827e3db046c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:53.218805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e98b48-e479-4d17-aa21-8827e3db046c admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:53.219444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e98b48-e479-4d17-aa21-8827e3db046c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:53.221031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e98b48-e479-4d17-aa21-8827e3db046c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:53.232459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1136/2278] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:15:53.236484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88685d01-bf87-4d21-b659-1d8db883c872 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:53.247534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88685d01-bf87-4d21-b659-1d8db883c872 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:53.252900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836144-1f9d-43a5-9262-254f752ddd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:53.253393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836144-1f9d-43a5-9262-254f752ddd0e admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:53.253672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836144-1f9d-43a5-9262-254f752ddd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:53.253943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836144-1f9d-43a5-9262-254f752ddd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:53.254613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9836144-1f9d-43a5-9262-254f752ddd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:53.256232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9836144-1f9d-43a5-9262-254f752ddd0e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:53.272041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1143/2279] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:15:53.273151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e98b48-e479-4d17-aa21-8827e3db046c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:53.284169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e98b48-e479-4d17-aa21-8827e3db046c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:53.289455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74019f-90de-4a8f-9ea6-c8c2e4dc3dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:53.290175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74019f-90de-4a8f-9ea6-c8c2e4dc3dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:53.290459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74019f-90de-4a8f-9ea6-c8c2e4dc3dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:53.290745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74019f-90de-4a8f-9ea6-c8c2e4dc3dcd admin admin] PATH_INFO: `/v3/projects/b722cde0a1b340c89a9c173edb401359/users/c20e157b546b4f66a975f55db0e44cc5/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:53.309705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74019f-90de-4a8f-9ea6-c8c2e4dc3dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b722cde0a1b340c89a9c173edb401359, user_id=c20e157b546b4f66a975f55db0e44cc5, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:53.312419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74019f-90de-4a8f-9ea6-c8c2e4dc3dcd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:53.419439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1137/2280] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:15:53 2022] PUT /identity/v3/projects/b722cde0a1b340c89a9c173edb401359/users/c20e157b546b4f66a975f55db0e44cc5/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:15:53.424576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48be05-0c00-4ad6-900b-dd8e7a77a584 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:53.424865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48be05-0c00-4ad6-900b-dd8e7a77a584 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:53.425138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48be05-0c00-4ad6-900b-dd8e7a77a584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:53.451511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ef48be05-0c00-4ad6-900b-dd8e7a77a584 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:15:53.452928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ef48be05-0c00-4ad6-900b-dd8e7a77a584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:53.531997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1144/2281] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:15:53.542304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74019f-90de-4a8f-9ea6-c8c2e4dc3dcd admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:53.544576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0a74019f-90de-4a8f-9ea6-c8c2e4dc3dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:53.559724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:15:53.603938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74019f-90de-4a8f-9ea6-c8c2e4dc3dcd admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:53.609158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdeb5a-dea8-4209-a1f4-6d045befafd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:53.609763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdeb5a-dea8-4209-a1f4-6d045befafd1 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:53.610223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdeb5a-dea8-4209-a1f4-6d045befafd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:53.610496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcdeb5a-dea8-4209-a1f4-6d045befafd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:53.621639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdeb5a-dea8-4209-a1f4-6d045befafd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:53.623100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcdeb5a-dea8-4209-a1f4-6d045befafd1 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:53.632716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1138/2282] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:15:55.327859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661919cf-3466-41b0-bf12-06f18f6f4246 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:55.328376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661919cf-3466-41b0-bf12-06f18f6f4246 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:55.328565 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661919cf-3466-41b0-bf12-06f18f6f4246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:55.357423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-661919cf-3466-41b0-bf12-06f18f6f4246 None None] MFA Rules not processed for user `c20e157b546b4f66a975f55db0e44cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:15:55.358783 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-661919cf-3466-41b0-bf12-06f18f6f4246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:55.415049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:15:55.424916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1145/2283] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:15:55 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:15:55.429634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdeb5a-dea8-4209-a1f4-6d045befafd1 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:55.439694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcdeb5a-dea8-4209-a1f4-6d045befafd1 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:55.444460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050155-caf9-464c-aeaa-065a73a8c8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:55.444943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050155-caf9-464c-aeaa-065a73a8c8ff admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:55.445220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050155-caf9-464c-aeaa-065a73a8c8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:55.445491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050155-caf9-464c-aeaa-065a73a8c8ff admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:55.446245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0050155-caf9-464c-aeaa-065a73a8c8ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:55.448162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0050155-caf9-464c-aeaa-065a73a8c8ff admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:55.584564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1139/2284] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:15:55 2022] POST /identity/v3/projects => generated 403 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:15:55.590517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661919cf-3466-41b0-bf12-06f18f6f4246 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:55.601809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661919cf-3466-41b0-bf12-06f18f6f4246 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:55.661537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c492b-61bb-4140-a621-9adc6a341fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:55.662285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c492b-61bb-4140-a621-9adc6a341fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:55.662441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c492b-61bb-4140-a621-9adc6a341fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:55.662710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890c492b-61bb-4140-a621-9adc6a341fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:55.663331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c492b-61bb-4140-a621-9adc6a341fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:55.665347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890c492b-61bb-4140-a621-9adc6a341fe8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:55.789084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1146/2285] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:15:55 2022] POST /identity/v3/users => generated 308 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:15:55.793022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050155-caf9-464c-aeaa-065a73a8c8ff admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:55.803291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050155-caf9-464c-aeaa-065a73a8c8ff admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:55.807850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fa79c-9659-4afe-b344-33afd8fdd717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:55.808929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fa79c-9659-4afe-b344-33afd8fdd717 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:55.809201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fa79c-9659-4afe-b344-33afd8fdd717 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:55.809467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fa79c-9659-4afe-b344-33afd8fdd717 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:55.810245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fa79c-9659-4afe-b344-33afd8fdd717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:55.811812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fa79c-9659-4afe-b344-33afd8fdd717 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:55.822216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1140/2286] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:15:55.830614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c492b-61bb-4140-a621-9adc6a341fe8 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:55.841465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890c492b-61bb-4140-a621-9adc6a341fe8 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:55.850546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8425366-ec90-45d2-b655-952fceae7f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:55.851287 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8425366-ec90-45d2-b655-952fceae7f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:55.851621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8425366-ec90-45d2-b655-952fceae7f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:55.851969 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8425366-ec90-45d2-b655-952fceae7f18 admin admin] PATH_INFO: `/v3/projects/e14064c2725848b1b349cdc705fda843/users/b04f400929f24dc7b65a468dc8bce92f/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:55.872554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8425366-ec90-45d2-b655-952fceae7f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14064c2725848b1b349cdc705fda843, user_id=b04f400929f24dc7b65a468dc8bce92f, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:55.875192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8425366-ec90-45d2-b655-952fceae7f18 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:55.961374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1147/2287] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:15:55 2022] PUT /identity/v3/projects/e14064c2725848b1b349cdc705fda843/users/b04f400929f24dc7b65a468dc8bce92f/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:15:55.966068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fa79c-9659-4afe-b344-33afd8fdd717 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:55.978380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fa79c-9659-4afe-b344-33afd8fdd717 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:56.041609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbb29f-860c-47d4-9177-35949b48cf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:56.042164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbb29f-860c-47d4-9177-35949b48cf49 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:56.042964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbb29f-860c-47d4-9177-35949b48cf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:56.043231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbb29f-860c-47d4-9177-35949b48cf49 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:56.043876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbb29f-860c-47d4-9177-35949b48cf49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:56.045508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbb29f-860c-47d4-9177-35949b48cf49 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:56.059425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1141/2288] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:15:56.063751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8425366-ec90-45d2-b655-952fceae7f18 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:56.074823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8425366-ec90-45d2-b655-952fceae7f18 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:56.079957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcde21-7192-4338-b54c-973b2906509d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:56.080633 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcde21-7192-4338-b54c-973b2906509d admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:56.080909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcde21-7192-4338-b54c-973b2906509d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:56.081181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdcde21-7192-4338-b54c-973b2906509d admin admin] PATH_INFO: `/v3/projects/e14064c2725848b1b349cdc705fda843/users/b04f400929f24dc7b65a468dc8bce92f/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:56.083109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcde21-7192-4338-b54c-973b2906509d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14064c2725848b1b349cdc705fda843, user_id=b04f400929f24dc7b65a468dc8bce92f, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:56.085745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdcde21-7192-4338-b54c-973b2906509d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:56.097540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1148/2289] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:15:56 2022] PUT /identity/v3/projects/e14064c2725848b1b349cdc705fda843/users/b04f400929f24dc7b65a468dc8bce92f/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:15:57.995879 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e7494-c1a2-4377-aff2-1bdeae81edbc None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:57.996458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e7494-c1a2-4377-aff2-1bdeae81edbc None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:57.996458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e7494-c1a2-4377-aff2-1bdeae81edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:58.024669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-897e7494-c1a2-4377-aff2-1bdeae81edbc None None] MFA Rules not processed for user `b04f400929f24dc7b65a468dc8bce92f`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:15:58.025891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-897e7494-c1a2-4377-aff2-1bdeae81edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:15:58.098386 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1142/2290] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:15:57 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:15:59.460030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcde21-7192-4338-b54c-973b2906509d admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:15:59.527156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdcde21-7192-4338-b54c-973b2906509d admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:15:59.531862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:15:59.532940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:15:59.532940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:15:59.533123 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] PATH_INFO: `/v3/users/c20e157b546b4f66a975f55db0e44cc5` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:15:59.534189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c20e157b546b4f66a975f55db0e44cc5)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:15:59.536183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:15:59.765735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c20e157b546b4f66a975f55db0e44cc5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:59.767455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:15:59.920195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c20e157b546b4f66a975f55db0e44cc5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:59.933921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c20e157b546b4f66a975f55db0e44cc5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:59.934728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c20e157b546b4f66a975f55db0e44cc5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:59.935418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c20e157b546b4f66a975f55db0e44cc5'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:15:59.951322 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1149/2291] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:15:59 2022] DELETE /identity/v3/users/c20e157b546b4f66a975f55db0e44cc5 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:00.212725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e7494-c1a2-4377-aff2-1bdeae81edbc None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:00.219274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:00.226770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e7494-c1a2-4377-aff2-1bdeae81edbc None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:00.285580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:00.286229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:00.286507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:00.286773 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] PATH_INFO: `/v3/projects/b722cde0a1b340c89a9c173edb401359` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:00.287883 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b722cde0a1b340c89a9c173edb401359)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:00.289799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:00.386649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b722cde0a1b340c89a9c173edb401359', 'request_id': 'req-32180211-328d-4d5c-9fcc-0417537d8955'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:00.451009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b722cde0a1b340c89a9c173edb401359', 'request_id': 'req-32180211-328d-4d5c-9fcc-0417537d8955'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:00.457560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b722cde0a1b340c89a9c173edb401359', 'request_id': 'req-32180211-328d-4d5c-9fcc-0417537d8955'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:00.469134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] The token cache is being invalidate because project b722cde0a1b340c89a9c173edb401359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:16:00.469465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:00.470184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:00.471723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1143/2292] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:16:00 2022] DELETE /identity/v3/projects/b722cde0a1b340c89a9c173edb401359 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:00.475903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:00.477361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:00.492688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08164d47-3e08-4f0c-af52-a90bd3e6d0a4 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:00.553660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:00.554347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:00.554626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:00.554894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] PATH_INFO: `/v3/users/b04f400929f24dc7b65a468dc8bce92f` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:00.556151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04f400929f24dc7b65a468dc8bce92f)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:00.558351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:00.771200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04f400929f24dc7b65a468dc8bce92f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:00.833751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04f400929f24dc7b65a468dc8bce92f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:00.850794 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04f400929f24dc7b65a468dc8bce92f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:00.851739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04f400929f24dc7b65a468dc8bce92f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:00.852413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-845a7f88-40e8-4b0c-bab1-5c8e5fd8ad8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04f400929f24dc7b65a468dc8bce92f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:00.866070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1150/2293] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:16:00 2022] DELETE /identity/v3/users/b04f400929f24dc7b65a468dc8bce92f => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:01.121081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:01.122771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:01.151241 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32180211-328d-4d5c-9fcc-0417537d8955 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:01.215034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:01.216442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:01.216722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:01.216988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] PATH_INFO: `/v3/projects/e14064c2725848b1b349cdc705fda843` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:01.218436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e14064c2725848b1b349cdc705fda843)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:01.220320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:01.374288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:01.382421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e14064c2725848b1b349cdc705fda843', 'request_id': 'req-0627da2a-2641-473e-a92c-37b52ce9d86f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:01.459783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e14064c2725848b1b349cdc705fda843', 'request_id': 'req-0627da2a-2641-473e-a92c-37b52ce9d86f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:01.466483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e14064c2725848b1b349cdc705fda843', 'request_id': 'req-0627da2a-2641-473e-a92c-37b52ce9d86f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:01.477081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] The token cache is being invalidate because project e14064c2725848b1b349cdc705fda843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:16:01.477363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:01.478027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0627da2a-2641-473e-a92c-37b52ce9d86f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:01.479995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1144/2294] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:16:01 2022] DELETE /identity/v3/projects/e14064c2725848b1b349cdc705fda843 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:01.484888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42fc3d-66f1-4006-8e78-b427ece16ee8 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:01.485635 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42fc3d-66f1-4006-8e78-b427ece16ee8 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:01.485730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42fc3d-66f1-4006-8e78-b427ece16ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:01.506263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ea42fc3d-66f1-4006-8e78-b427ece16ee8 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:16:01.507499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ea42fc3d-66f1-4006-8e78-b427ece16ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:01.550720 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:01.570184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1151/2295] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:16:01.577899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21aa52f-70c3-43c9-a98e-7b6394b7f3f0 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:01.579831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21aa52f-70c3-43c9-a98e-7b6394b7f3f0 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:01.579831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21aa52f-70c3-43c9-a98e-7b6394b7f3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:01.598821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a21aa52f-70c3-43c9-a98e-7b6394b7f3f0 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:16:01.600021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a21aa52f-70c3-43c9-a98e-7b6394b7f3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:01.609040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1145/2296] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:16:01.614343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42fc3d-66f1-4006-8e78-b427ece16ee8 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:01.624045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42fc3d-66f1-4006-8e78-b427ece16ee8 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:01.628822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66e7ba-5378-49d1-b58c-60ca9169bddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:01.629350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66e7ba-5378-49d1-b58c-60ca9169bddc admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:01.629827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66e7ba-5378-49d1-b58c-60ca9169bddc admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:01.630143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66e7ba-5378-49d1-b58c-60ca9169bddc admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:01.630925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66e7ba-5378-49d1-b58c-60ca9169bddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:16:01.631212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66e7ba-5378-49d1-b58c-60ca9169bddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:01.632839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66e7ba-5378-49d1-b58c-60ca9169bddc admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:01.646172 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1152/2297] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:16:01.650825 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21aa52f-70c3-43c9-a98e-7b6394b7f3f0 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:01.660881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21aa52f-70c3-43c9-a98e-7b6394b7f3f0 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:01.665815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca01341-cfac-4b41-8dff-1614702e5c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:01.666548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01341-cfac-4b41-8dff-1614702e5c71 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:01.667606 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01341-cfac-4b41-8dff-1614702e5c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:01.667894 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca01341-cfac-4b41-8dff-1614702e5c71 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:01.668528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca01341-cfac-4b41-8dff-1614702e5c71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:01.670482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca01341-cfac-4b41-8dff-1614702e5c71 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:01.761415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1146/2298] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:16:01 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:16:01.765734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66e7ba-5378-49d1-b58c-60ca9169bddc admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:01.777500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66e7ba-5378-49d1-b58c-60ca9169bddc admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:01.836664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a51a4a-d2f9-4089-918f-3b1c1ea6b673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:01.837267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a51a4a-d2f9-4089-918f-3b1c1ea6b673 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:01.837543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a51a4a-d2f9-4089-918f-3b1c1ea6b673 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:01.837816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a51a4a-d2f9-4089-918f-3b1c1ea6b673 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:01.838459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a51a4a-d2f9-4089-918f-3b1c1ea6b673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:01.840849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a51a4a-d2f9-4089-918f-3b1c1ea6b673 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:01.961584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1153/2299] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:16:01 2022] POST /identity/v3/users => generated 283 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:16:01.962986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca01341-cfac-4b41-8dff-1614702e5c71 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:01.977675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca01341-cfac-4b41-8dff-1614702e5c71 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:01.984074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df50ff-7aec-4fba-85bd-6c541bf5122c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:01.984719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df50ff-7aec-4fba-85bd-6c541bf5122c admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:01.985032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df50ff-7aec-4fba-85bd-6c541bf5122c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:01.985610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df50ff-7aec-4fba-85bd-6c541bf5122c admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:01.986593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df50ff-7aec-4fba-85bd-6c541bf5122c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:01.988238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df50ff-7aec-4fba-85bd-6c541bf5122c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:01.998540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1147/2300] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:16:02.003349 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a51a4a-d2f9-4089-918f-3b1c1ea6b673 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:02.014589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a51a4a-d2f9-4089-918f-3b1c1ea6b673 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:02.019283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c184dd-6118-4e37-9b39-2929fb9b443d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:02.019880 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c184dd-6118-4e37-9b39-2929fb9b443d admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:02.020155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c184dd-6118-4e37-9b39-2929fb9b443d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:02.020425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c184dd-6118-4e37-9b39-2929fb9b443d admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:02.021061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c184dd-6118-4e37-9b39-2929fb9b443d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:02.023503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c184dd-6118-4e37-9b39-2929fb9b443d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:02.034586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1154/2301] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:16:02.038734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df50ff-7aec-4fba-85bd-6c541bf5122c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:02.049278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df50ff-7aec-4fba-85bd-6c541bf5122c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:02.054054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e8767-b77e-4537-8134-e6f721f1b324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:02.054695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e8767-b77e-4537-8134-e6f721f1b324 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:02.054972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e8767-b77e-4537-8134-e6f721f1b324 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:02.055241 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e8767-b77e-4537-8134-e6f721f1b324 admin admin] PATH_INFO: `/v3/projects/d3d7976a4b6b4b15b738abe8977b34e8/users/e3957f27f1874b369f4141737b939ab1/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:02.070099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e8767-b77e-4537-8134-e6f721f1b324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d7976a4b6b4b15b738abe8977b34e8, user_id=e3957f27f1874b369f4141737b939ab1, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:02.072728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e8767-b77e-4537-8134-e6f721f1b324 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:02.132550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1148/2302] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:16:02 2022] PUT /identity/v3/projects/d3d7976a4b6b4b15b738abe8977b34e8/users/e3957f27f1874b369f4141737b939ab1/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:02.153338 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41ef8-9c6d-42e6-815a-6d78e8cb00b9 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:02.153338 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41ef8-9c6d-42e6-815a-6d78e8cb00b9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:02.153338 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41ef8-9c6d-42e6-815a-6d78e8cb00b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:02.175736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-53f41ef8-9c6d-42e6-815a-6d78e8cb00b9 None None] MFA Rules not processed for user `e3957f27f1874b369f4141737b939ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:16:02.176946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-53f41ef8-9c6d-42e6-815a-6d78e8cb00b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:02.243185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1155/2303] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:16:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:16:02.251542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e8767-b77e-4537-8134-e6f721f1b324 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:02.314648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e8767-b77e-4537-8134-e6f721f1b324 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:02.319437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2aadc8-09c5-4f02-8235-762148d6f130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:02.320052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aadc8-09c5-4f02-8235-762148d6f130 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:02.320312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aadc8-09c5-4f02-8235-762148d6f130 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:02.320582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2aadc8-09c5-4f02-8235-762148d6f130 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:02.321224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2aadc8-09c5-4f02-8235-762148d6f130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:02.323627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2aadc8-09c5-4f02-8235-762148d6f130 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:02.431578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1149/2304] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:16:02 2022] POST /identity/v3/projects => generated 365 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:16:02.436327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41ef8-9c6d-42e6-815a-6d78e8cb00b9 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:02.449654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41ef8-9c6d-42e6-815a-6d78e8cb00b9 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:02.491704 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:02.522518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0def155-fee3-454f-9c9d-ecbb9d4c59db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:02.523291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0def155-fee3-454f-9c9d-ecbb9d4c59db admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:02.523563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0def155-fee3-454f-9c9d-ecbb9d4c59db admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:02.523831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0def155-fee3-454f-9c9d-ecbb9d4c59db admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:02.524464 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0def155-fee3-454f-9c9d-ecbb9d4c59db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:02.528500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0def155-fee3-454f-9c9d-ecbb9d4c59db admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:02.619715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1156/2305] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:16:02 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:16:02.624644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2aadc8-09c5-4f02-8235-762148d6f130 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:02.635346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2aadc8-09c5-4f02-8235-762148d6f130 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:02.641580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5c67e-5ab2-4112-a099-9192a56b5439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:02.641580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5c67e-5ab2-4112-a099-9192a56b5439 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:02.644023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5c67e-5ab2-4112-a099-9192a56b5439 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:02.644023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5c67e-5ab2-4112-a099-9192a56b5439 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:02.644023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5c67e-5ab2-4112-a099-9192a56b5439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:02.644368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5c67e-5ab2-4112-a099-9192a56b5439 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:02.655927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1150/2306] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:16:02.662905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0def155-fee3-454f-9c9d-ecbb9d4c59db admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:02.674361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0def155-fee3-454f-9c9d-ecbb9d4c59db admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:02.679824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb40af-ba4b-48ed-8f1d-c297845abb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:02.680489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb40af-ba4b-48ed-8f1d-c297845abb42 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:02.680773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb40af-ba4b-48ed-8f1d-c297845abb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:02.681132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb40af-ba4b-48ed-8f1d-c297845abb42 admin admin] PATH_INFO: `/v3/projects/9b7ee0ddd6024bc0b9e2b4dded697635/users/2865189a9e4b47d5a07b3b1010ed0857/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:02.700172 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb40af-ba4b-48ed-8f1d-c297845abb42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7ee0ddd6024bc0b9e2b4dded697635, user_id=2865189a9e4b47d5a07b3b1010ed0857, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:02.702782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb40af-ba4b-48ed-8f1d-c297845abb42 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:02.769829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1157/2307] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:16:02 2022] PUT /identity/v3/projects/9b7ee0ddd6024bc0b9e2b4dded697635/users/2865189a9e4b47d5a07b3b1010ed0857/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:02.773913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5c67e-5ab2-4112-a099-9192a56b5439 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:02.785072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5c67e-5ab2-4112-a099-9192a56b5439 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:02.821531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:02.845798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd1e31-b215-45ec-b852-21a87106480a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:02.846425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd1e31-b215-45ec-b852-21a87106480a admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:02.846704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd1e31-b215-45ec-b852-21a87106480a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:02.847129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd1e31-b215-45ec-b852-21a87106480a admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:02.847888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd1e31-b215-45ec-b852-21a87106480a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:02.849518 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd1e31-b215-45ec-b852-21a87106480a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:02.860154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1151/2308] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:16:02.864661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb40af-ba4b-48ed-8f1d-c297845abb42 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:02.876488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb40af-ba4b-48ed-8f1d-c297845abb42 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:02.881090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2862d7b-5b27-44db-8f63-341d4aeeac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:02.882142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2862d7b-5b27-44db-8f63-341d4aeeac71 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:02.882142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2862d7b-5b27-44db-8f63-341d4aeeac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:02.882474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2862d7b-5b27-44db-8f63-341d4aeeac71 admin admin] PATH_INFO: `/v3/projects/9b7ee0ddd6024bc0b9e2b4dded697635/users/2865189a9e4b47d5a07b3b1010ed0857/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:02.884233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2862d7b-5b27-44db-8f63-341d4aeeac71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7ee0ddd6024bc0b9e2b4dded697635, user_id=2865189a9e4b47d5a07b3b1010ed0857, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:02.887049 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2862d7b-5b27-44db-8f63-341d4aeeac71 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:02.897750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1158/2309] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:16:02 2022] PUT /identity/v3/projects/9b7ee0ddd6024bc0b9e2b4dded697635/users/2865189a9e4b47d5a07b3b1010ed0857/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:02.908588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e12b6c-2f85-40b8-a575-fde01d67365a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:02.908877 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e12b6c-2f85-40b8-a575-fde01d67365a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:02.909153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e12b6c-2f85-40b8-a575-fde01d67365a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:02.939545 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c1e12b6c-2f85-40b8-a575-fde01d67365a None None] MFA Rules not processed for user `2865189a9e4b47d5a07b3b1010ed0857`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:16:02.940760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c1e12b6c-2f85-40b8-a575-fde01d67365a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:03.034194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1152/2310] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:16:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:16:03.045734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2862d7b-5b27-44db-8f63-341d4aeeac71 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:03.047551 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f2862d7b-5b27-44db-8f63-341d4aeeac71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:03.117810 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2862d7b-5b27-44db-8f63-341d4aeeac71 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:03.123886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09025ab-b727-4f14-91c5-5acae386c0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:03.124558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09025ab-b727-4f14-91c5-5acae386c0dd service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:03.124846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09025ab-b727-4f14-91c5-5acae386c0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:03.125112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09025ab-b727-4f14-91c5-5acae386c0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:03.134524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09025ab-b727-4f14-91c5-5acae386c0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:03.136069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09025ab-b727-4f14-91c5-5acae386c0dd service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:03.145912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1159/2311] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:16:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:16:03.334965 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e12b6c-2f85-40b8-a575-fde01d67365a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:03.345431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e12b6c-2f85-40b8-a575-fde01d67365a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:03.349307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733dc2dd-6720-4499-b00f-ad8eb21079ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:03.349904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dc2dd-6720-4499-b00f-ad8eb21079ba service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:03.350206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dc2dd-6720-4499-b00f-ad8eb21079ba service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:03.350497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733dc2dd-6720-4499-b00f-ad8eb21079ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:03.407048 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dc2dd-6720-4499-b00f-ad8eb21079ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:03.408537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733dc2dd-6720-4499-b00f-ad8eb21079ba service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:03.417872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1153/2312] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:16:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:16:07.080257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09025ab-b727-4f14-91c5-5acae386c0dd service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:07.142455 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09025ab-b727-4f14-91c5-5acae386c0dd service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:07.147116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:07.147734 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:07.148006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:07.148278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] PATH_INFO: `/v3/users/e3957f27f1874b369f4141737b939ab1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:07.149427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3957f27f1874b369f4141737b939ab1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:07.151896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:07.249140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:07.317342 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3957f27f1874b369f4141737b939ab1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:07.391048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3957f27f1874b369f4141737b939ab1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:07.409258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3957f27f1874b369f4141737b939ab1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:07.410058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3957f27f1874b369f4141737b939ab1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:07.410695 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3957f27f1874b369f4141737b939ab1'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:07.424632 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1160/2313] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:16:07 2022] DELETE /identity/v3/users/e3957f27f1874b369f4141737b939ab1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:07.429445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219ddc9-ffbf-4904-9a3d-69cc68191f50 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:07.429943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219ddc9-ffbf-4904-9a3d-69cc68191f50 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:07.430254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219ddc9-ffbf-4904-9a3d-69cc68191f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:07.451291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5219ddc9-ffbf-4904-9a3d-69cc68191f50 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:16:07.452627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5219ddc9-ffbf-4904-9a3d-69cc68191f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:07.507588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:07.520032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1154/2314] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:16:07.532589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:07.534966 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:07.613795 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1767a-e12d-4c3d-90e9-278e1ef5a967 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:07.617969 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174d03b-e504-4284-a41c-b326c2323a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:07.623882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174d03b-e504-4284-a41c-b326c2323a06 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:07.623882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174d03b-e504-4284-a41c-b326c2323a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:07.623882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174d03b-e504-4284-a41c-b326c2323a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:07.629424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174d03b-e504-4284-a41c-b326c2323a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:07.630875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7174d03b-e504-4284-a41c-b326c2323a06 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:07.645211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1161/2315] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:16:08.138153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219ddc9-ffbf-4904-9a3d-69cc68191f50 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:08.139601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5219ddc9-ffbf-4904-9a3d-69cc68191f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:08.152953 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219ddc9-ffbf-4904-9a3d-69cc68191f50 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:08.157663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:08.158389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:08.158697 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:08.158972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] PATH_INFO: `/v3/projects/d3d7976a4b6b4b15b738abe8977b34e8` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:08.160103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d7976a4b6b4b15b738abe8977b34e8)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:08.162022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:08.266362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d7976a4b6b4b15b738abe8977b34e8', 'request_id': 'req-100cbb28-cbf4-4bee-a54f-a46387f3cb84'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:08.347924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d7976a4b6b4b15b738abe8977b34e8', 'request_id': 'req-100cbb28-cbf4-4bee-a54f-a46387f3cb84'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:08.354930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d7976a4b6b4b15b738abe8977b34e8', 'request_id': 'req-100cbb28-cbf4-4bee-a54f-a46387f3cb84'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:08.366501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] The token cache is being invalidate because project d3d7976a4b6b4b15b738abe8977b34e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:16:08.366775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:08.367427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:08.368916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1155/2316] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:16:08 2022] DELETE /identity/v3/projects/d3d7976a4b6b4b15b738abe8977b34e8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:08.373618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174d03b-e504-4284-a41c-b326c2323a06 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:08.383613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174d03b-e504-4284-a41c-b326c2323a06 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:08.450979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:08.451609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:08.451885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:08.452153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] PATH_INFO: `/v3/users/2865189a9e4b47d5a07b3b1010ed0857` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:08.453269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2865189a9e4b47d5a07b3b1010ed0857)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:08.455195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:08.642398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2865189a9e4b47d5a07b3b1010ed0857'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:08.644160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:08.803148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2865189a9e4b47d5a07b3b1010ed0857'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:08.826412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2865189a9e4b47d5a07b3b1010ed0857'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:08.827768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2865189a9e4b47d5a07b3b1010ed0857'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:08.830409 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-013d4542-dbd1-4992-b2f0-b860028d43d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2865189a9e4b47d5a07b3b1010ed0857'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:08.843492 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1162/2317] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:16:08 2022] DELETE /identity/v3/users/2865189a9e4b47d5a07b3b1010ed0857 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:09.361457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:09.362955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:09.376910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cbb28-cbf4-4bee-a54f-a46387f3cb84 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:09.442487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:09.443112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:09.443436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:09.443694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] PATH_INFO: `/v3/projects/9b7ee0ddd6024bc0b9e2b4dded697635` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:09.444820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b7ee0ddd6024bc0b9e2b4dded697635)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:09.447154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:09.607330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b7ee0ddd6024bc0b9e2b4dded697635', 'request_id': 'req-a89d3b63-275b-4d5b-9694-3923971eaafe'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:09.690708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b7ee0ddd6024bc0b9e2b4dded697635', 'request_id': 'req-a89d3b63-275b-4d5b-9694-3923971eaafe'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:09.697485 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b7ee0ddd6024bc0b9e2b4dded697635', 'request_id': 'req-a89d3b63-275b-4d5b-9694-3923971eaafe'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:09.709325 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] The token cache is being invalidate because project 9b7ee0ddd6024bc0b9e2b4dded697635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:16:09.709526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:09.710235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a89d3b63-275b-4d5b-9694-3923971eaafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:16:09.711729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1156/2318] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:16:09 2022] DELETE /identity/v3/projects/9b7ee0ddd6024bc0b9e2b4dded697635 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:09.716670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72654169-ccca-4b2d-90bf-ca73e9a4d80e None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:09.717137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72654169-ccca-4b2d-90bf-ca73e9a4d80e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:09.717225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72654169-ccca-4b2d-90bf-ca73e9a4d80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:09.737539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-72654169-ccca-4b2d-90bf-ca73e9a4d80e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:16:09.738771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-72654169-ccca-4b2d-90bf-ca73e9a4d80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:09.797669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1163/2319] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:16:09.805401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181360fb-74a1-410a-bb2a-4055f9ee18ac None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:09.805689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181360fb-74a1-410a-bb2a-4055f9ee18ac None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:09.805996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181360fb-74a1-410a-bb2a-4055f9ee18ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:09.826109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-181360fb-74a1-410a-bb2a-4055f9ee18ac None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:16:09.827364 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-181360fb-74a1-410a-bb2a-4055f9ee18ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:09.835488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1157/2320] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:16:09.840361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72654169-ccca-4b2d-90bf-ca73e9a4d80e None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:09.850048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72654169-ccca-4b2d-90bf-ca73e9a4d80e None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:09.855265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a8a6a-a210-418a-ae20-d3dcfc52352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:09.855851 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a8a6a-a210-418a-ae20-d3dcfc52352e admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:09.856147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a8a6a-a210-418a-ae20-d3dcfc52352e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:09.856396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a8a6a-a210-418a-ae20-d3dcfc52352e admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:09.857060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a8a6a-a210-418a-ae20-d3dcfc52352e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:16:09.857358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a8a6a-a210-418a-ae20-d3dcfc52352e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:09.858955 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a8a6a-a210-418a-ae20-d3dcfc52352e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:09.870817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1164/2321] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:16:09.875346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181360fb-74a1-410a-bb2a-4055f9ee18ac None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:09.879931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:09.885396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181360fb-74a1-410a-bb2a-4055f9ee18ac None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:09.890245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd6a16-284b-4f97-b723-498da4558755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:09.890861 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd6a16-284b-4f97-b723-498da4558755 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:09.891136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd6a16-284b-4f97-b723-498da4558755 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:09.891405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd6a16-284b-4f97-b723-498da4558755 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:09.892030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd6a16-284b-4f97-b723-498da4558755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:09.893919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd6a16-284b-4f97-b723-498da4558755 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:09.993369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1158/2322] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:16:09 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:16:09.997891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a8a6a-a210-418a-ae20-d3dcfc52352e admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:10.013977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a8a6a-a210-418a-ae20-d3dcfc52352e admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:10.078351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68c8c4-4b89-4314-b929-c740e9d6687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:10.078958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68c8c4-4b89-4314-b929-c740e9d6687f admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:10.079235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68c8c4-4b89-4314-b929-c740e9d6687f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:10.079503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68c8c4-4b89-4314-b929-c740e9d6687f admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:10.080121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68c8c4-4b89-4314-b929-c740e9d6687f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:10.082075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68c8c4-4b89-4314-b929-c740e9d6687f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:10.178314 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1165/2323] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:16:09 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:16:10.182620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd6a16-284b-4f97-b723-498da4558755 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:10.194055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd6a16-284b-4f97-b723-498da4558755 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:10.198889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24884b8-b67a-42e5-81ad-3021092af92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:10.199474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24884b8-b67a-42e5-81ad-3021092af92e admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:10.199746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24884b8-b67a-42e5-81ad-3021092af92e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:10.200010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24884b8-b67a-42e5-81ad-3021092af92e admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:10.200639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24884b8-b67a-42e5-81ad-3021092af92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:10.202239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24884b8-b67a-42e5-81ad-3021092af92e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:10.212028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1159/2324] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:16:10.216479 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68c8c4-4b89-4314-b929-c740e9d6687f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:10.226584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68c8c4-4b89-4314-b929-c740e9d6687f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:10.232336 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce1ef4-27ef-4023-a848-b325b72480d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:10.232945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce1ef4-27ef-4023-a848-b325b72480d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:10.233222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce1ef4-27ef-4023-a848-b325b72480d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:10.233494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce1ef4-27ef-4023-a848-b325b72480d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:10.234173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce1ef4-27ef-4023-a848-b325b72480d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:10.236026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce1ef4-27ef-4023-a848-b325b72480d7 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:10.247038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1166/2325] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:16:10.251292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24884b8-b67a-42e5-81ad-3021092af92e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:10.261702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24884b8-b67a-42e5-81ad-3021092af92e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:10.267121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3919350-153e-4558-b043-2f3185f6f3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:10.267754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3919350-153e-4558-b043-2f3185f6f3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:10.268031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3919350-153e-4558-b043-2f3185f6f3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:10.268302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3919350-153e-4558-b043-2f3185f6f3d6 admin admin] PATH_INFO: `/v3/projects/cb0622e26cc5416c8a507c20c88f6cc6/users/6c5f09d5c2864bb6b5251018e05c31ed/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:10.283771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3919350-153e-4558-b043-2f3185f6f3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0622e26cc5416c8a507c20c88f6cc6, user_id=6c5f09d5c2864bb6b5251018e05c31ed, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:10.286436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3919350-153e-4558-b043-2f3185f6f3d6 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:10.410899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1160/2326] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:16:10 2022] PUT /identity/v3/projects/cb0622e26cc5416c8a507c20c88f6cc6/users/6c5f09d5c2864bb6b5251018e05c31ed/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:10.422460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0f79-fb5a-4dfd-b2d9-7278748ddc0d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:10.422822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0f79-fb5a-4dfd-b2d9-7278748ddc0d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:10.423234 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b0f79-fb5a-4dfd-b2d9-7278748ddc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:10.451786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-3a7b0f79-fb5a-4dfd-b2d9-7278748ddc0d None None] MFA Rules not processed for user `6c5f09d5c2864bb6b5251018e05c31ed`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:16:10.453021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3a7b0f79-fb5a-4dfd-b2d9-7278748ddc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:10.472052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:10.518940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1167/2327] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:16:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:16:10.529123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3919350-153e-4558-b043-2f3185f6f3d6 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:10.593135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3919350-153e-4558-b043-2f3185f6f3d6 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:10.597819 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed12ad-d9fc-4141-ad29-c7e69454cae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:10.598441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed12ad-d9fc-4141-ad29-c7e69454cae9 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:10.598721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed12ad-d9fc-4141-ad29-c7e69454cae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:10.598994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed12ad-d9fc-4141-ad29-c7e69454cae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:10.599612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed12ad-d9fc-4141-ad29-c7e69454cae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:10.601490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed12ad-d9fc-4141-ad29-c7e69454cae9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:10.724053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1161/2328] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:16:10 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:16:10.728186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b0f79-fb5a-4dfd-b2d9-7278748ddc0d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:10.739228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b0f79-fb5a-4dfd-b2d9-7278748ddc0d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:10.801024 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62f286-fff1-4403-a918-781f48375097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:10.801518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62f286-fff1-4403-a918-781f48375097 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:10.801797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62f286-fff1-4403-a918-781f48375097 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:10.802110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62f286-fff1-4403-a918-781f48375097 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:10.802746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62f286-fff1-4403-a918-781f48375097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:10.804669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62f286-fff1-4403-a918-781f48375097 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:10.894188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1168/2329] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:16:10 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:16:10.898352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed12ad-d9fc-4141-ad29-c7e69454cae9 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:10.909653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed12ad-d9fc-4141-ad29-c7e69454cae9 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:10.914417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aae71b-752e-475b-bf4b-2c82a24217f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:10.915065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aae71b-752e-475b-bf4b-2c82a24217f4 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:10.915365 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aae71b-752e-475b-bf4b-2c82a24217f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:10.915636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aae71b-752e-475b-bf4b-2c82a24217f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:10.916310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aae71b-752e-475b-bf4b-2c82a24217f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:10.918479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aae71b-752e-475b-bf4b-2c82a24217f4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:10.929522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1162/2330] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:16:10.933819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62f286-fff1-4403-a918-781f48375097 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:10.946340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62f286-fff1-4403-a918-781f48375097 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:10.951314 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16680a-ae95-44f5-97e4-1d10e3045f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:10.952001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16680a-ae95-44f5-97e4-1d10e3045f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:10.952457 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16680a-ae95-44f5-97e4-1d10e3045f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:10.952729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16680a-ae95-44f5-97e4-1d10e3045f02 admin admin] PATH_INFO: `/v3/projects/41404ea74a4c4acc894511223e5ed2a9/users/ec1225ec397f43cb821e4516dacbc6e0/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:10.968848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16680a-ae95-44f5-97e4-1d10e3045f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41404ea74a4c4acc894511223e5ed2a9, user_id=ec1225ec397f43cb821e4516dacbc6e0, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:10.971488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16680a-ae95-44f5-97e4-1d10e3045f02 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:11.049043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1169/2331] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:16:10 2022] PUT /identity/v3/projects/41404ea74a4c4acc894511223e5ed2a9/users/ec1225ec397f43cb821e4516dacbc6e0/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:11.053364 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aae71b-752e-475b-bf4b-2c82a24217f4 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:11.063650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aae71b-752e-475b-bf4b-2c82a24217f4 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:11.126586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e4289b-da59-4d44-aee6-2b6004b8cc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:11.127183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4289b-da59-4d44-aee6-2b6004b8cc5b admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:11.127459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4289b-da59-4d44-aee6-2b6004b8cc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:11.127728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e4289b-da59-4d44-aee6-2b6004b8cc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:11.128370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4289b-da59-4d44-aee6-2b6004b8cc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:11.129973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e4289b-da59-4d44-aee6-2b6004b8cc5b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:11.140680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1163/2332] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:16:11.144775 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16680a-ae95-44f5-97e4-1d10e3045f02 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:11.155867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16680a-ae95-44f5-97e4-1d10e3045f02 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:11.160466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0faa8-6852-43b9-8ab8-05d248974973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:11.162368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0faa8-6852-43b9-8ab8-05d248974973 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:11.162368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0faa8-6852-43b9-8ab8-05d248974973 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:11.162368 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0faa8-6852-43b9-8ab8-05d248974973 admin admin] PATH_INFO: `/v3/projects/41404ea74a4c4acc894511223e5ed2a9/users/ec1225ec397f43cb821e4516dacbc6e0/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:11.163567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0faa8-6852-43b9-8ab8-05d248974973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41404ea74a4c4acc894511223e5ed2a9, user_id=ec1225ec397f43cb821e4516dacbc6e0, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:11.166517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0faa8-6852-43b9-8ab8-05d248974973 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:11.176793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1170/2333] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:16:11 2022] PUT /identity/v3/projects/41404ea74a4c4acc894511223e5ed2a9/users/ec1225ec397f43cb821e4516dacbc6e0/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:16:11.189302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa6dbf-b31f-4362-aa9e-f6eb43a5ccc6 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:11.191122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa6dbf-b31f-4362-aa9e-f6eb43a5ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:11.191122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa6dbf-b31f-4362-aa9e-f6eb43a5ccc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:11.224735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-61aa6dbf-b31f-4362-aa9e-f6eb43a5ccc6 None None] MFA Rules not processed for user `ec1225ec397f43cb821e4516dacbc6e0`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:16:11.225938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-61aa6dbf-b31f-4362-aa9e-f6eb43a5ccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:11.230407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:11.304510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1164/2334] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:16:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:16:11.315499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0faa8-6852-43b9-8ab8-05d248974973 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:11.317325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-25b0faa8-6852-43b9-8ab8-05d248974973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:11.386680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0faa8-6852-43b9-8ab8-05d248974973 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:11.391289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563860f8-d6b5-4466-bc64-239b63f7e6d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:11.391910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563860f8-d6b5-4466-bc64-239b63f7e6d8 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:11.392186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563860f8-d6b5-4466-bc64-239b63f7e6d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:11.392453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563860f8-d6b5-4466-bc64-239b63f7e6d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:11.403804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563860f8-d6b5-4466-bc64-239b63f7e6d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:11.405277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563860f8-d6b5-4466-bc64-239b63f7e6d8 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:11.416383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1171/2335] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:16:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:16:11.615995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa6dbf-b31f-4362-aa9e-f6eb43a5ccc6 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:11.626736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa6dbf-b31f-4362-aa9e-f6eb43a5ccc6 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:11.630259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08182d-44fa-4b41-84e7-714f0feef3a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:11.630846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08182d-44fa-4b41-84e7-714f0feef3a8 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:11.631118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08182d-44fa-4b41-84e7-714f0feef3a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:11.631581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08182d-44fa-4b41-84e7-714f0feef3a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:11.695889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08182d-44fa-4b41-84e7-714f0feef3a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:11.697298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08182d-44fa-4b41-84e7-714f0feef3a8 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:11.706787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1165/2336] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:16:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:16:21.411440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:21.702784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=55 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:40.451942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563860f8-d6b5-4466-bc64-239b63f7e6d8 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:40.459653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-563860f8-d6b5-4466-bc64-239b63f7e6d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:40.575622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563860f8-d6b5-4466-bc64-239b63f7e6d8 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:40.584006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66c980-79eb-4e3c-adc1-e241560f5413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:40.584949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66c980-79eb-4e3c-adc1-e241560f5413 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:40.585239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66c980-79eb-4e3c-adc1-e241560f5413 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:40.587514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66c980-79eb-4e3c-adc1-e241560f5413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:40.587808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-7c66c980-79eb-4e3c-adc1-e241560f5413 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:40.611127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66c980-79eb-4e3c-adc1-e241560f5413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:40.612571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66c980-79eb-4e3c-adc1-e241560f5413 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:40.628644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1172/2337] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:16:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:16:41.330420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08182d-44fa-4b41-84e7-714f0feef3a8 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:41.347754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08182d-44fa-4b41-84e7-714f0feef3a8 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:41.355138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aa155-5adc-441a-bcd7-5f4f4f6bdab7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:41.356986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aa155-5adc-441a-bcd7-5f4f4f6bdab7 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:41.358025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aa155-5adc-441a-bcd7-5f4f4f6bdab7 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:41.359033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2aa155-5adc-441a-bcd7-5f4f4f6bdab7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:41.363457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ea2aa155-5adc-441a-bcd7-5f4f4f6bdab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:41.382828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2aa155-5adc-441a-bcd7-5f4f4f6bdab7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:41.385307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2aa155-5adc-441a-bcd7-5f4f4f6bdab7 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:41.399161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1166/2338] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:16:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:16:44.071081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7e627-fd8c-48e8-9f65-23e0be294d6d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:44.071081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7e627-fd8c-48e8-9f65-23e0be294d6d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:44.071081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7e627-fd8c-48e8-9f65-23e0be294d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:44.101175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f1a7e627-fd8c-48e8-9f65-23e0be294d6d None None] MFA Rules not processed for user `6b7457ab864e4df6b10a195f8500c14c`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:16:44.102401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f1a7e627-fd8c-48e8-9f65-23e0be294d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:44.164392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1173/2339] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:16:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:16:44.704997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aa155-5adc-441a-bcd7-5f4f4f6bdab7 service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:44.706430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ea2aa155-5adc-441a-bcd7-5f4f4f6bdab7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:44.762850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2aa155-5adc-441a-bcd7-5f4f4f6bdab7 service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:44.766441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac3d62-b4bc-4411-a9ec-35816b765085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:44.767032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac3d62-b4bc-4411-a9ec-35816b765085 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:44.767307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac3d62-b4bc-4411-a9ec-35816b765085 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:44.767593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac3d62-b4bc-4411-a9ec-35816b765085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:44.769272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-49ac3d62-b4bc-4411-a9ec-35816b765085 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:44.778720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac3d62-b4bc-4411-a9ec-35816b765085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:44.780122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac3d62-b4bc-4411-a9ec-35816b765085 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:44.787909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1167/2340] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:16:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:16:46.221066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7e627-fd8c-48e8-9f65-23e0be294d6d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:46.222480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f1a7e627-fd8c-48e8-9f65-23e0be294d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:46.290987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7e627-fd8c-48e8-9f65-23e0be294d6d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:46.296256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2a959-360d-4cb2-a0d0-c5772906e972 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:46.296861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2a959-360d-4cb2-a0d0-c5772906e972 service placement] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:46.297139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2a959-360d-4cb2-a0d0-c5772906e972 service placement] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:46.297406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2a959-360d-4cb2-a0d0-c5772906e972 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:46.299284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2ce2a959-360d-4cb2-a0d0-c5772906e972 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:46.311535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2a959-360d-4cb2-a0d0-c5772906e972 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:46.313079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2a959-360d-4cb2-a0d0-c5772906e972 service placement] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:46.323562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1174/2341] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:16:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:16:54.784427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=32 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:16:55.255740 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac3d62-b4bc-4411-a9ec-35816b765085 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:16:55.257294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-49ac3d62-b4bc-4411-a9ec-35816b765085 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:55.273038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac3d62-b4bc-4411-a9ec-35816b765085 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:16:55.279349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143db5b5-a9f3-4db0-8c71-db559e08d5c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:16:55.279940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143db5b5-a9f3-4db0-8c71-db559e08d5c3 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:16:55.280217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143db5b5-a9f3-4db0-8c71-db559e08d5c3 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:16:55.280488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143db5b5-a9f3-4db0-8c71-db559e08d5c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:16:55.282196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-143db5b5-a9f3-4db0-8c71-db559e08d5c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:16:55.373850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143db5b5-a9f3-4db0-8c71-db559e08d5c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:16:55.375580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143db5b5-a9f3-4db0-8c71-db559e08d5c3 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:16:55.396546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1168/2342] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:16:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:16:56.319190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=94 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:17:00.190231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2a959-360d-4cb2-a0d0-c5772906e972 service placement] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:00.204722 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2a959-360d-4cb2-a0d0-c5772906e972 service placement] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:00.210342 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3ae93-5bf7-47a7-bb87-313f624901b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:00.210935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3ae93-5bf7-47a7-bb87-313f624901b9 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:00.211237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3ae93-5bf7-47a7-bb87-313f624901b9 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:00.211504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac3ae93-5bf7-47a7-bb87-313f624901b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:00.213188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-dac3ae93-5bf7-47a7-bb87-313f624901b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:00.226476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3ae93-5bf7-47a7-bb87-313f624901b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:00.228048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac3ae93-5bf7-47a7-bb87-313f624901b9 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:00.238930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1175/2343] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:17:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:17:05.393764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=41 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:17:06.949776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3680db-d5a5-4059-ab2f-3b946a37d7d9 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:06.950395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3680db-d5a5-4059-ab2f-3b946a37d7d9 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:06.950395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3680db-d5a5-4059-ab2f-3b946a37d7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:07.008248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-bf3680db-d5a5-4059-ab2f-3b946a37d7d9 None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:17:07.009457 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bf3680db-d5a5-4059-ab2f-3b946a37d7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:07.079830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1169/2344] 10.222.0.79 () {58 vars in 967 bytes} [Tue Aug 9 17:17:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:17:07.234384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3ae93-5bf7-47a7-bb87-313f624901b9 service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:07.235771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-dac3ae93-5bf7-47a7-bb87-313f624901b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:07.323278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac3ae93-5bf7-47a7-bb87-313f624901b9 service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:07.328139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9261c24-c784-4909-9f9a-579764eb7c79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:07.328730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9261c24-c784-4909-9f9a-579764eb7c79 service swift] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:07.329001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9261c24-c784-4909-9f9a-579764eb7c79 service swift] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:07.329263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9261c24-c784-4909-9f9a-579764eb7c79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:07.337091 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9261c24-c784-4909-9f9a-579764eb7c79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:07.338684 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9261c24-c784-4909-9f9a-579764eb7c79 service swift] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:07.346436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1176/2345] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 17:17:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:17:08.707191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3680db-d5a5-4059-ab2f-3b946a37d7d9 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:08.708618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bf3680db-d5a5-4059-ab2f-3b946a37d7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:08.726409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3680db-d5a5-4059-ab2f-3b946a37d7d9 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:08.726652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2adb1-3815-40af-935c-c2da7119fb27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:08.727265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2adb1-3815-40af-935c-c2da7119fb27 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:08.727546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2adb1-3815-40af-935c-c2da7119fb27 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:08.727818 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2adb1-3815-40af-935c-c2da7119fb27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:08.729737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-17e2adb1-3815-40af-935c-c2da7119fb27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:08.800859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2adb1-3815-40af-935c-c2da7119fb27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:08.802340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2adb1-3815-40af-935c-c2da7119fb27 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:08.812256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1170/2346] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:17:11.365516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fded1-2b3c-43c4-8a25-d516699c0dd3 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:11.366015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fded1-2b3c-43c4-8a25-d516699c0dd3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:11.366169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775fded1-2b3c-43c4-8a25-d516699c0dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:11.384938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-775fded1-2b3c-43c4-8a25-d516699c0dd3 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:17:11.386154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-775fded1-2b3c-43c4-8a25-d516699c0dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:11.393733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1177/2347] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:17:11.402596 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b2d9-3cac-49af-9ffe-3706b843450a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:11.403095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b2d9-3cac-49af-9ffe-3706b843450a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:11.403404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556b2d9-3cac-49af-9ffe-3706b843450a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:11.423561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-b556b2d9-3cac-49af-9ffe-3706b843450a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:17:11.424639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-b556b2d9-3cac-49af-9ffe-3706b843450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:11.432657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1171/2348] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:17:11.437415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fded1-2b3c-43c4-8a25-d516699c0dd3 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:11.447655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775fded1-2b3c-43c4-8a25-d516699c0dd3 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:11.452347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50643e26-26ac-4c56-a027-355f1b6918a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:11.452960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50643e26-26ac-4c56-a027-355f1b6918a3 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:11.453238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50643e26-26ac-4c56-a027-355f1b6918a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:11.453509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50643e26-26ac-4c56-a027-355f1b6918a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:11.454213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50643e26-26ac-4c56-a027-355f1b6918a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:17:11.454516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50643e26-26ac-4c56-a027-355f1b6918a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:11.456098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50643e26-26ac-4c56-a027-355f1b6918a3 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:11.469506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1178/2349] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:17:11.474051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556b2d9-3cac-49af-9ffe-3706b843450a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:11.484869 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556b2d9-3cac-49af-9ffe-3706b843450a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:11.489812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88027730-65e1-4d78-ade3-7b184df8ce82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:11.490468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88027730-65e1-4d78-ade3-7b184df8ce82 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:11.490772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88027730-65e1-4d78-ade3-7b184df8ce82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:11.491038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88027730-65e1-4d78-ade3-7b184df8ce82 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:11.491674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88027730-65e1-4d78-ade3-7b184df8ce82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:11.493584 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88027730-65e1-4d78-ade3-7b184df8ce82 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:11.618213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1172/2350] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:17:11 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:17:11.619781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50643e26-26ac-4c56-a027-355f1b6918a3 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:11.630232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50643e26-26ac-4c56-a027-355f1b6918a3 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:11.691894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a8092-39c3-4bc0-aab6-1fcd0938e59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:11.692502 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a8092-39c3-4bc0-aab6-1fcd0938e59e admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:11.692784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a8092-39c3-4bc0-aab6-1fcd0938e59e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:11.693052 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a8092-39c3-4bc0-aab6-1fcd0938e59e admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:11.693710 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a8092-39c3-4bc0-aab6-1fcd0938e59e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:11.695666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a8092-39c3-4bc0-aab6-1fcd0938e59e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:11.795789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1179/2351] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:17:11 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:17:11.800097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88027730-65e1-4d78-ade3-7b184df8ce82 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:11.810247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88027730-65e1-4d78-ade3-7b184df8ce82 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:11.814847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c839d65-06cc-48a0-8632-6c169f9b69bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:11.815451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c839d65-06cc-48a0-8632-6c169f9b69bd admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:11.815724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c839d65-06cc-48a0-8632-6c169f9b69bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:11.815995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c839d65-06cc-48a0-8632-6c169f9b69bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:11.816640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c839d65-06cc-48a0-8632-6c169f9b69bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:11.818349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c839d65-06cc-48a0-8632-6c169f9b69bd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:11.828727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1173/2352] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:17:11.832775 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a8092-39c3-4bc0-aab6-1fcd0938e59e admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:11.844084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a8092-39c3-4bc0-aab6-1fcd0938e59e admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:11.849346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e0247-457d-4745-a79b-d3171c92da3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:11.850313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e0247-457d-4745-a79b-d3171c92da3b admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:11.850590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e0247-457d-4745-a79b-d3171c92da3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:11.850855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e0247-457d-4745-a79b-d3171c92da3b admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:11.851487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e0247-457d-4745-a79b-d3171c92da3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:11.853100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e0247-457d-4745-a79b-d3171c92da3b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:11.864983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1180/2353] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:17:11.869263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c839d65-06cc-48a0-8632-6c169f9b69bd admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:11.879254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c839d65-06cc-48a0-8632-6c169f9b69bd admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:11.884374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945d03e-1e38-4c61-b159-fee8aabc7000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:11.885003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d03e-1e38-4c61-b159-fee8aabc7000 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:11.885282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d03e-1e38-4c61-b159-fee8aabc7000 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:11.885554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d03e-1e38-4c61-b159-fee8aabc7000 admin admin] PATH_INFO: `/v3/projects/cb9a129996ad4cbd9e5e14b7dfac224e/users/1daa219840c84738a11f07cee8991b27/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:11.900618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945d03e-1e38-4c61-b159-fee8aabc7000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb9a129996ad4cbd9e5e14b7dfac224e, user_id=1daa219840c84738a11f07cee8991b27, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:11.903429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945d03e-1e38-4c61-b159-fee8aabc7000 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:11.905647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:17:11.972843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1174/2354] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:17:11 2022] PUT /identity/v3/projects/cb9a129996ad4cbd9e5e14b7dfac224e/users/1daa219840c84738a11f07cee8991b27/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:17:11.978367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9631a-4875-4ed3-ac49-a4c5ec2be58c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:11.978657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9631a-4875-4ed3-ac49-a4c5ec2be58c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:11.978925 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9631a-4875-4ed3-ac49-a4c5ec2be58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:11.998177 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-5fc9631a-4875-4ed3-ac49-a4c5ec2be58c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:17:11.999435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5fc9631a-4875-4ed3-ac49-a4c5ec2be58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:12.010045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:17:12.061675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1181/2355] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:17:12.072159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945d03e-1e38-4c61-b159-fee8aabc7000 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:12.128669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945d03e-1e38-4c61-b159-fee8aabc7000 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:12.133054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d111b-9e7a-488d-bcc3-736bd403b950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:12.133827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d111b-9e7a-488d-bcc3-736bd403b950 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:12.133979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d111b-9e7a-488d-bcc3-736bd403b950 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:12.134210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d111b-9e7a-488d-bcc3-736bd403b950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:12.143116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d111b-9e7a-488d-bcc3-736bd403b950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:12.144539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d111b-9e7a-488d-bcc3-736bd403b950 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:12.153801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1175/2356] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:17:12.533577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9631a-4875-4ed3-ac49-a4c5ec2be58c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:12.535442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5fc9631a-4875-4ed3-ac49-a4c5ec2be58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:12.593029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9631a-4875-4ed3-ac49-a4c5ec2be58c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:12.596743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bad2d-e7ab-43be-a65c-4c558bd0ef7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:12.597356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bad2d-e7ab-43be-a65c-4c558bd0ef7c service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:12.597616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bad2d-e7ab-43be-a65c-4c558bd0ef7c service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:12.598024 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0bad2d-e7ab-43be-a65c-4c558bd0ef7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:12.600255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5c0bad2d-e7ab-43be-a65c-4c558bd0ef7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:12.691480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bad2d-e7ab-43be-a65c-4c558bd0ef7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:12.693029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0bad2d-e7ab-43be-a65c-4c558bd0ef7c service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:12.712836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1182/2357] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:17:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:17:17.489913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac6a22-bdcd-4316-854f-58d676ddc028 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:17.490958 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac6a22-bdcd-4316-854f-58d676ddc028 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:17.491083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac6a22-bdcd-4316-854f-58d676ddc028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:17.535075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-01ac6a22-bdcd-4316-854f-58d676ddc028 None None] MFA Rules not processed for user `1daa219840c84738a11f07cee8991b27`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:17:17.536456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-01ac6a22-bdcd-4316-854f-58d676ddc028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:17.636231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1176/2358] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:17:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:17:17.643488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bad2d-e7ab-43be-a65c-4c558bd0ef7c service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:17.676571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0bad2d-e7ab-43be-a65c-4c558bd0ef7c service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:17.684363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9558e-a6fd-40d3-aff1-76a5648ca283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:17.684974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9558e-a6fd-40d3-aff1-76a5648ca283 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:17.685248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9558e-a6fd-40d3-aff1-76a5648ca283 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:17.685510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9558e-a6fd-40d3-aff1-76a5648ca283 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:17.686164 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9558e-a6fd-40d3-aff1-76a5648ca283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:17.688060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9558e-a6fd-40d3-aff1-76a5648ca283 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:17.719667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:17:17.794974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1183/2359] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:17:17 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:17:17.799323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac6a22-bdcd-4316-854f-58d676ddc028 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:17.811779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac6a22-bdcd-4316-854f-58d676ddc028 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:17.889213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0967-7921-4393-8800-4aa3fd93c11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:17.890238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0967-7921-4393-8800-4aa3fd93c11a admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:17.890708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0967-7921-4393-8800-4aa3fd93c11a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:17.891165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0967-7921-4393-8800-4aa3fd93c11a admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:17.891964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c0967-7921-4393-8800-4aa3fd93c11a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:17.894040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c0967-7921-4393-8800-4aa3fd93c11a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:17.997037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1177/2360] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:17:17 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:17:18.002352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9558e-a6fd-40d3-aff1-76a5648ca283 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:18.015936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9558e-a6fd-40d3-aff1-76a5648ca283 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:18.025017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f15192-6ec9-4820-8f59-3f973585fcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:18.025641 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f15192-6ec9-4820-8f59-3f973585fcfa admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:18.026204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f15192-6ec9-4820-8f59-3f973585fcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:18.026204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f15192-6ec9-4820-8f59-3f973585fcfa admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:18.027335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f15192-6ec9-4820-8f59-3f973585fcfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:18.030334 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f15192-6ec9-4820-8f59-3f973585fcfa admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:18.050330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1184/2361] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:17:18.055495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0967-7921-4393-8800-4aa3fd93c11a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:18.071528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0967-7921-4393-8800-4aa3fd93c11a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:18.077580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb04208-3e18-4fd9-b976-6db4f0e754ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:18.078336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb04208-3e18-4fd9-b976-6db4f0e754ee admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:18.078615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb04208-3e18-4fd9-b976-6db4f0e754ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:18.078890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb04208-3e18-4fd9-b976-6db4f0e754ee admin admin] PATH_INFO: `/v3/projects/40a4c8b97a9b418e8a332bfe5253c95a/users/cc5b9a2b52004f948caf529d9f47ea68/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:18.095320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:17:18.098711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb04208-3e18-4fd9-b976-6db4f0e754ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a4c8b97a9b418e8a332bfe5253c95a, user_id=cc5b9a2b52004f948caf529d9f47ea68, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:18.105797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb04208-3e18-4fd9-b976-6db4f0e754ee admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:18.189814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1178/2362] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:17:18 2022] PUT /identity/v3/projects/40a4c8b97a9b418e8a332bfe5253c95a/users/cc5b9a2b52004f948caf529d9f47ea68/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:17:18.193979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f15192-6ec9-4820-8f59-3f973585fcfa admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:18.204563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f15192-6ec9-4820-8f59-3f973585fcfa admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:18.278680 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74f812-6eef-487a-9108-e15c587a0338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:18.279450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74f812-6eef-487a-9108-e15c587a0338 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:18.279761 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74f812-6eef-487a-9108-e15c587a0338 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:18.280028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74f812-6eef-487a-9108-e15c587a0338 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:18.280664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74f812-6eef-487a-9108-e15c587a0338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:18.282486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c74f812-6eef-487a-9108-e15c587a0338 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:18.293507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1185/2363] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:17:18.297818 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb04208-3e18-4fd9-b976-6db4f0e754ee admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:18.308881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb04208-3e18-4fd9-b976-6db4f0e754ee admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:18.313783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24583feb-06f3-49a3-9a65-21c44af2e747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:18.319243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24583feb-06f3-49a3-9a65-21c44af2e747 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:18.319243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24583feb-06f3-49a3-9a65-21c44af2e747 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:18.319243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24583feb-06f3-49a3-9a65-21c44af2e747 admin admin] PATH_INFO: `/v3/projects/40a4c8b97a9b418e8a332bfe5253c95a/users/cc5b9a2b52004f948caf529d9f47ea68/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:18.319243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24583feb-06f3-49a3-9a65-21c44af2e747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a4c8b97a9b418e8a332bfe5253c95a, user_id=cc5b9a2b52004f948caf529d9f47ea68, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:18.319593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24583feb-06f3-49a3-9a65-21c44af2e747 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:18.330552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1179/2364] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:17:18 2022] PUT /identity/v3/projects/40a4c8b97a9b418e8a332bfe5253c95a/users/cc5b9a2b52004f948caf529d9f47ea68/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:17:23.886921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88908e-5125-4dfa-9481-eddb0144b32b None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:23.887404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88908e-5125-4dfa-9481-eddb0144b32b None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:23.887404 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88908e-5125-4dfa-9481-eddb0144b32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:23.928242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ae88908e-5125-4dfa-9481-eddb0144b32b None None] MFA Rules not processed for user `cc5b9a2b52004f948caf529d9f47ea68`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:17:23.929463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ae88908e-5125-4dfa-9481-eddb0144b32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:24.004231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1186/2365] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:17:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:17:24.018013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24583feb-06f3-49a3-9a65-21c44af2e747 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:24.108267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24583feb-06f3-49a3-9a65-21c44af2e747 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:24.114934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fe5d-bb3e-4040-9e89-98521754c09d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:24.115650 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fe5d-bb3e-4040-9e89-98521754c09d service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:24.115925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fe5d-bb3e-4040-9e89-98521754c09d service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:24.116201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fe5d-bb3e-4040-9e89-98521754c09d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:24.186274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16fe5d-bb3e-4040-9e89-98521754c09d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:24.187874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16fe5d-bb3e-4040-9e89-98521754c09d service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:24.198117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1180/2366] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:17:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:17:26.983853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac941313-a2a0-4fc9-88e5-6015bd4db4a6 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:26.984505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac941313-a2a0-4fc9-88e5-6015bd4db4a6 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:26.984618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac941313-a2a0-4fc9-88e5-6015bd4db4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:27.012721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ac941313-a2a0-4fc9-88e5-6015bd4db4a6 None None] MFA Rules not processed for user `1daa219840c84738a11f07cee8991b27`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:17:27.014839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ac941313-a2a0-4fc9-88e5-6015bd4db4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:27.030903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1187/2367] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:17:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:17:34.193175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:17:37.004285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=78 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:17:54.618757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fe5d-bb3e-4040-9e89-98521754c09d service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:54.680930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fe5d-bb3e-4040-9e89-98521754c09d service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:54.684531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ba4be-24c7-4d4f-8abf-06d3a3d50fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:54.685125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ba4be-24c7-4d4f-8abf-06d3a3d50fff service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:54.685402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ba4be-24c7-4d4f-8abf-06d3a3d50fff service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:54.685669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ba4be-24c7-4d4f-8abf-06d3a3d50fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:54.687439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-229ba4be-24c7-4d4f-8abf-06d3a3d50fff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:54.697214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ba4be-24c7-4d4f-8abf-06d3a3d50fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:54.698681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ba4be-24c7-4d4f-8abf-06d3a3d50fff service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:54.708258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1181/2368] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:17:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:17:56.631835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac941313-a2a0-4fc9-88e5-6015bd4db4a6 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:17:56.646704 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac941313-a2a0-4fc9-88e5-6015bd4db4a6 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:17:56.650395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c29a5-0e5d-4d31-9189-14aa6e482903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:17:56.650992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c29a5-0e5d-4d31-9189-14aa6e482903 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:17:56.651270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c29a5-0e5d-4d31-9189-14aa6e482903 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:17:56.651537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c29a5-0e5d-4d31-9189-14aa6e482903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:17:56.653267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-470c29a5-0e5d-4d31-9189-14aa6e482903 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:17:56.667874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c29a5-0e5d-4d31-9189-14aa6e482903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:17:56.669532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c29a5-0e5d-4d31-9189-14aa6e482903 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:17:56.680779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1188/2369] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:17:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:18:04.705357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=26 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:18:06.675293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=6 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:18:31.056608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e9851-8596-4c01-85c7-84db455f5f47 None None] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:31.059706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e9851-8596-4c01-85c7-84db455f5f47 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:31.059706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e9851-8596-4c01-85c7-84db455f5f47 None None] PATH_INFO: `/` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:31.059706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1182/2370] 10.222.0.79 () {54 vars in 921 bytes} [Tue Aug 9 17:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 09 17:18:31.086493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4b477-654c-4642-8663-669a093684dd None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:31.087011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4b477-654c-4642-8663-669a093684dd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:31.087011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4b477-654c-4642-8663-669a093684dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:31.126224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-77c4b477-654c-4642-8663-669a093684dd None None] MFA Rules not processed for user `d7bc938254514471801c88761525e883`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:18:31.128105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-77c4b477-654c-4642-8663-669a093684dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:31.206612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1189/2371] 10.222.0.79 () {58 vars in 1018 bytes} [Tue Aug 9 17:18:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:18:31.234826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e9851-8596-4c01-85c7-84db455f5f47 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:31.254674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e9851-8596-4c01-85c7-84db455f5f47 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:31.269239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76853fd-31d0-4a5d-be33-2e431df246f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:31.270286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76853fd-31d0-4a5d-be33-2e431df246f2 service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:31.270817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76853fd-31d0-4a5d-be33-2e431df246f2 service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:31.271282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76853fd-31d0-4a5d-be33-2e431df246f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:31.281393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76853fd-31d0-4a5d-be33-2e431df246f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:31.283177 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76853fd-31d0-4a5d-be33-2e431df246f2 service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:31.294076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1183/2372] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:18:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:18:34.971396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4b477-654c-4642-8663-669a093684dd None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:35.045754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4b477-654c-4642-8663-669a093684dd None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:35.050829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:35.051534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:35.051900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:35.052258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] PATH_INFO: `/v3/users/1daa219840c84738a11f07cee8991b27` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:35.053476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1daa219840c84738a11f07cee8991b27)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:35.055564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:35.227163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1daa219840c84738a11f07cee8991b27'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:35.326743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1daa219840c84738a11f07cee8991b27'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:35.340770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1daa219840c84738a11f07cee8991b27'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:35.341430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1daa219840c84738a11f07cee8991b27'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:35.342056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1daa219840c84738a11f07cee8991b27'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:35.355068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1190/2373] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:18:34 2022] DELETE /identity/v3/users/1daa219840c84738a11f07cee8991b27 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:18:35.626556 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76853fd-31d0-4a5d-be33-2e431df246f2 service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:35.639224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76853fd-31d0-4a5d-be33-2e431df246f2 service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:35.702933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:35.703548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:35.703816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:35.704096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] PATH_INFO: `/v3/projects/cb9a129996ad4cbd9e5e14b7dfac224e` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:35.705208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb9a129996ad4cbd9e5e14b7dfac224e)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:35.707389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:35.830861 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb9a129996ad4cbd9e5e14b7dfac224e', 'request_id': 'req-e3e588e6-2431-4ff1-b061-4948b8169d4b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:35.898699 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb9a129996ad4cbd9e5e14b7dfac224e', 'request_id': 'req-e3e588e6-2431-4ff1-b061-4948b8169d4b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:35.905333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb9a129996ad4cbd9e5e14b7dfac224e', 'request_id': 'req-e3e588e6-2431-4ff1-b061-4948b8169d4b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:35.917104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] The token cache is being invalidate because project cb9a129996ad4cbd9e5e14b7dfac224e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:18:35.917388 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:35.918095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:35.919660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1184/2374] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:18:35 2022] DELETE /identity/v3/projects/cb9a129996ad4cbd9e5e14b7dfac224e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:18:35.923726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:35.925206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:35.933778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:18:35.939217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8333e-e545-4ef0-89a0-552452c4ac9f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:35.998327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:35.999026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:35.999207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:35.999475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] PATH_INFO: `/v3/users/cc5b9a2b52004f948caf529d9f47ea68` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:36.000590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc5b9a2b52004f948caf529d9f47ea68)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:36.002572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:36.142715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc5b9a2b52004f948caf529d9f47ea68'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:36.196400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc5b9a2b52004f948caf529d9f47ea68'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:36.209365 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc5b9a2b52004f948caf529d9f47ea68'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:36.210131 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc5b9a2b52004f948caf529d9f47ea68'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:36.210768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-678972ba-4411-4bc8-a4ec-4e452c04b90f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc5b9a2b52004f948caf529d9f47ea68'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:36.225025 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1191/2375] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:18:35 2022] DELETE /identity/v3/users/cc5b9a2b52004f948caf529d9f47ea68 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:18:36.479713 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:36.481234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:36.494921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e588e6-2431-4ff1-b061-4948b8169d4b admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:36.571458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:36.572072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:36.572349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:36.572619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] PATH_INFO: `/v3/projects/40a4c8b97a9b418e8a332bfe5253c95a` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:36.573724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a4c8b97a9b418e8a332bfe5253c95a)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:36.575686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:36.596403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:18:36.700919 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a4c8b97a9b418e8a332bfe5253c95a', 'request_id': 'req-1861dca1-b452-4d56-a8d5-674f53a624cf'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:36.784069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a4c8b97a9b418e8a332bfe5253c95a', 'request_id': 'req-1861dca1-b452-4d56-a8d5-674f53a624cf'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:36.791711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a4c8b97a9b418e8a332bfe5253c95a', 'request_id': 'req-1861dca1-b452-4d56-a8d5-674f53a624cf'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:36.803867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] The token cache is being invalidate because project 40a4c8b97a9b418e8a332bfe5253c95a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:18:36.804425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:36.805211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1861dca1-b452-4d56-a8d5-674f53a624cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:18:36.807129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1185/2376] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:18:36 2022] DELETE /identity/v3/projects/40a4c8b97a9b418e8a332bfe5253c95a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:18:36.812139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342a937-f451-4adf-a5a5-6f9f0f0dd2c9 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:36.812426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342a937-f451-4adf-a5a5-6f9f0f0dd2c9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:36.812954 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342a937-f451-4adf-a5a5-6f9f0f0dd2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:36.865055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c342a937-f451-4adf-a5a5-6f9f0f0dd2c9 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:18:36.866345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c342a937-f451-4adf-a5a5-6f9f0f0dd2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:36.917303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:18:36.925567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1192/2377] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:18:36.939598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ea506-b05b-4810-8e1c-83b5f9efd96a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:36.939882 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ea506-b05b-4810-8e1c-83b5f9efd96a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:36.940156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ea506-b05b-4810-8e1c-83b5f9efd96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:36.959554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c98ea506-b05b-4810-8e1c-83b5f9efd96a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:18:36.961280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c98ea506-b05b-4810-8e1c-83b5f9efd96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:36.970062 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1186/2378] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:18:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:18:36.977169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342a937-f451-4adf-a5a5-6f9f0f0dd2c9 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:36.986784 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342a937-f451-4adf-a5a5-6f9f0f0dd2c9 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:36.991878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0595698-9d2d-4519-ba93-400f0359a220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:36.992465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0595698-9d2d-4519-ba93-400f0359a220 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:36.992740 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0595698-9d2d-4519-ba93-400f0359a220 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:36.993010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0595698-9d2d-4519-ba93-400f0359a220 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:36.993642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0595698-9d2d-4519-ba93-400f0359a220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:18:36.993941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0595698-9d2d-4519-ba93-400f0359a220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:36.995571 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0595698-9d2d-4519-ba93-400f0359a220 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:37.007652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1193/2379] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:18:37.011967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ea506-b05b-4810-8e1c-83b5f9efd96a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:37.021551 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ea506-b05b-4810-8e1c-83b5f9efd96a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:37.026719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9e65b-f324-434a-a304-4b64c3c3c09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:37.027307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9e65b-f324-434a-a304-4b64c3c3c09c admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:37.027580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9e65b-f324-434a-a304-4b64c3c3c09c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:37.027845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9e65b-f324-434a-a304-4b64c3c3c09c admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:37.028470 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9e65b-f324-434a-a304-4b64c3c3c09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:37.030443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9e65b-f324-434a-a304-4b64c3c3c09c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:37.105983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1187/2380] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:18:37 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:18:37.109981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0595698-9d2d-4519-ba93-400f0359a220 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:37.120792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0595698-9d2d-4519-ba93-400f0359a220 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:37.176339 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae046acd-70f3-44b2-9fa6-aed5e2249f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:37.176937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae046acd-70f3-44b2-9fa6-aed5e2249f22 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:37.177213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae046acd-70f3-44b2-9fa6-aed5e2249f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:37.177478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae046acd-70f3-44b2-9fa6-aed5e2249f22 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:37.178132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae046acd-70f3-44b2-9fa6-aed5e2249f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:37.180311 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae046acd-70f3-44b2-9fa6-aed5e2249f22 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:37.287071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1194/2381] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:18:37 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:18:37.291264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9e65b-f324-434a-a304-4b64c3c3c09c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:37.302070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9e65b-f324-434a-a304-4b64c3c3c09c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:37.306816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35da3e-359d-4496-9b2a-3cb8be9796b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:37.307402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35da3e-359d-4496-9b2a-3cb8be9796b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:37.307809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35da3e-359d-4496-9b2a-3cb8be9796b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:37.308113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35da3e-359d-4496-9b2a-3cb8be9796b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:37.308753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35da3e-359d-4496-9b2a-3cb8be9796b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:37.310483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35da3e-359d-4496-9b2a-3cb8be9796b2 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:37.321401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1188/2382] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:18:37.325326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae046acd-70f3-44b2-9fa6-aed5e2249f22 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:37.335996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae046acd-70f3-44b2-9fa6-aed5e2249f22 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:37.340687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416d8cb-67a4-43f6-ae29-08a4457dc9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:37.341274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416d8cb-67a4-43f6-ae29-08a4457dc9ec admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:37.341552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416d8cb-67a4-43f6-ae29-08a4457dc9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:37.341821 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416d8cb-67a4-43f6-ae29-08a4457dc9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:37.342496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416d8cb-67a4-43f6-ae29-08a4457dc9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:37.344063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416d8cb-67a4-43f6-ae29-08a4457dc9ec admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:37.354267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1195/2383] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:18:37.358591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35da3e-359d-4496-9b2a-3cb8be9796b2 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:37.368228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35da3e-359d-4496-9b2a-3cb8be9796b2 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:37.372741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7e576-6829-4394-9dd1-092eb84c6d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:37.373362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7e576-6829-4394-9dd1-092eb84c6d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:37.373636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7e576-6829-4394-9dd1-092eb84c6d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:37.373903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7e576-6829-4394-9dd1-092eb84c6d90 admin admin] PATH_INFO: `/v3/projects/7ee36e36177140ccb933e3d6f4f46b3c/users/99a24283bfde46b0872b1c4a10e8d617/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:37.388444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7e576-6829-4394-9dd1-092eb84c6d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ee36e36177140ccb933e3d6f4f46b3c, user_id=99a24283bfde46b0872b1c4a10e8d617, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:37.391091 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7e576-6829-4394-9dd1-092eb84c6d90 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:37.499286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1189/2384] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:18:37 2022] PUT /identity/v3/projects/7ee36e36177140ccb933e3d6f4f46b3c/users/99a24283bfde46b0872b1c4a10e8d617/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:18:37.504429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903274-6b0e-4539-bc7d-563f437c460d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:37.504713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903274-6b0e-4539-bc7d-563f437c460d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:37.504992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60903274-6b0e-4539-bc7d-563f437c460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:37.524456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-60903274-6b0e-4539-bc7d-563f437c460d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:18:37.525648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-60903274-6b0e-4539-bc7d-563f437c460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:37.585133 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1196/2385] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:18:37.594774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7e576-6829-4394-9dd1-092eb84c6d90 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:37.596575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bca7e576-6829-4394-9dd1-092eb84c6d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:37.653808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7e576-6829-4394-9dd1-092eb84c6d90 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:37.657986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679e9f1-ff75-43f6-b89a-8b96bea2429f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:37.658586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679e9f1-ff75-43f6-b89a-8b96bea2429f service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:37.658862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679e9f1-ff75-43f6-b89a-8b96bea2429f service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:37.659132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679e9f1-ff75-43f6-b89a-8b96bea2429f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:37.668160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679e9f1-ff75-43f6-b89a-8b96bea2429f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:37.669569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679e9f1-ff75-43f6-b89a-8b96bea2429f service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:37.679289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1190/2386] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:18:41.895192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60903274-6b0e-4539-bc7d-563f437c460d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:41.897882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-60903274-6b0e-4539-bc7d-563f437c460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:41.981932 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60903274-6b0e-4539-bc7d-563f437c460d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:41.986103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb281258-9ffe-4876-bde2-ad655694138d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:41.986702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb281258-9ffe-4876-bde2-ad655694138d service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:41.986977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb281258-9ffe-4876-bde2-ad655694138d service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:41.987242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb281258-9ffe-4876-bde2-ad655694138d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:41.988952 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-cb281258-9ffe-4876-bde2-ad655694138d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:42.012390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:18:42.080396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb281258-9ffe-4876-bde2-ad655694138d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:42.082147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb281258-9ffe-4876-bde2-ad655694138d service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:42.104825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1197/2387] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:18:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:18:43.316164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5dd0f-9466-4fba-bf25-aec48ccbfcc1 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:43.316808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5dd0f-9466-4fba-bf25-aec48ccbfcc1 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:43.317054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5dd0f-9466-4fba-bf25-aec48ccbfcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:43.354040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a2d5dd0f-9466-4fba-bf25-aec48ccbfcc1 None None] MFA Rules not processed for user `99a24283bfde46b0872b1c4a10e8d617`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:18:43.355307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a2d5dd0f-9466-4fba-bf25-aec48ccbfcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:43.398103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:18:43.437904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1191/2388] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:18:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:18:43.445532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb281258-9ffe-4876-bde2-ad655694138d service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:43.453846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb281258-9ffe-4876-bde2-ad655694138d service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:43.458675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e060b-113d-46c7-9283-53f5405f9ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:43.459281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e060b-113d-46c7-9283-53f5405f9ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:43.460154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e060b-113d-46c7-9283-53f5405f9ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:43.460471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e060b-113d-46c7-9283-53f5405f9ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:43.461083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e060b-113d-46c7-9283-53f5405f9ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:43.463056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e060b-113d-46c7-9283-53f5405f9ec5 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:43.583031 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1198/2389] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:18:43 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:18:43.590271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5dd0f-9466-4fba-bf25-aec48ccbfcc1 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:43.603493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5dd0f-9466-4fba-bf25-aec48ccbfcc1 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:43.714393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ac77a-7ab5-4ed2-8e22-cefecc2f646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:43.715573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ac77a-7ab5-4ed2-8e22-cefecc2f646b admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:43.716156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ac77a-7ab5-4ed2-8e22-cefecc2f646b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:43.716722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ac77a-7ab5-4ed2-8e22-cefecc2f646b admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:43.717625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ac77a-7ab5-4ed2-8e22-cefecc2f646b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:43.719785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ac77a-7ab5-4ed2-8e22-cefecc2f646b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:43.865048 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1192/2390] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:18:43 2022] POST /identity/v3/users => generated 296 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:18:43.869241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e060b-113d-46c7-9283-53f5405f9ec5 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:43.881897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e060b-113d-46c7-9283-53f5405f9ec5 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:43.886958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72815c08-707d-44c4-9fbf-8bc37858fd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:43.887992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72815c08-707d-44c4-9fbf-8bc37858fd23 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:43.888256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72815c08-707d-44c4-9fbf-8bc37858fd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:43.888506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72815c08-707d-44c4-9fbf-8bc37858fd23 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:43.889157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72815c08-707d-44c4-9fbf-8bc37858fd23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:43.891044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72815c08-707d-44c4-9fbf-8bc37858fd23 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:43.904048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1199/2391] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:18:43.908451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ac77a-7ab5-4ed2-8e22-cefecc2f646b admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:43.919306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ac77a-7ab5-4ed2-8e22-cefecc2f646b admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:43.925085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1103c-a7e7-4a6b-9a5a-dfd3a002ed3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:43.925689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1103c-a7e7-4a6b-9a5a-dfd3a002ed3a admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:43.926050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1103c-a7e7-4a6b-9a5a-dfd3a002ed3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:43.926365 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a1103c-a7e7-4a6b-9a5a-dfd3a002ed3a admin admin] PATH_INFO: `/v3/projects/4cb7628373d148baa468e2cabfd9fccb/users/ff544aeee2e14b32807ccfbdd03f8fb9/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:43.946469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a1103c-a7e7-4a6b-9a5a-dfd3a002ed3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb7628373d148baa468e2cabfd9fccb, user_id=ff544aeee2e14b32807ccfbdd03f8fb9, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:43.949187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a1103c-a7e7-4a6b-9a5a-dfd3a002ed3a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:44.028921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1193/2392] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:18:43 2022] PUT /identity/v3/projects/4cb7628373d148baa468e2cabfd9fccb/users/ff544aeee2e14b32807ccfbdd03f8fb9/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:18:44.032681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72815c08-707d-44c4-9fbf-8bc37858fd23 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:44.043499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72815c08-707d-44c4-9fbf-8bc37858fd23 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:44.144438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29d2d7-1d6d-4fab-b9cd-c7d35b40d554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:44.145378 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29d2d7-1d6d-4fab-b9cd-c7d35b40d554 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:44.145914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29d2d7-1d6d-4fab-b9cd-c7d35b40d554 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:44.146391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29d2d7-1d6d-4fab-b9cd-c7d35b40d554 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:44.147772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29d2d7-1d6d-4fab-b9cd-c7d35b40d554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:44.150276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29d2d7-1d6d-4fab-b9cd-c7d35b40d554 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:44.162353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1200/2393] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:18:44.166669 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1103c-a7e7-4a6b-9a5a-dfd3a002ed3a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:44.178004 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a1103c-a7e7-4a6b-9a5a-dfd3a002ed3a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:44.185044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0c17f-7de5-4248-b7a4-cb2fc5c8ee24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:44.185797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0c17f-7de5-4248-b7a4-cb2fc5c8ee24 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:44.186181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0c17f-7de5-4248-b7a4-cb2fc5c8ee24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:44.186518 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0c17f-7de5-4248-b7a4-cb2fc5c8ee24 admin admin] PATH_INFO: `/v3/projects/4cb7628373d148baa468e2cabfd9fccb/users/ff544aeee2e14b32807ccfbdd03f8fb9/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:44.188844 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0c17f-7de5-4248-b7a4-cb2fc5c8ee24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb7628373d148baa468e2cabfd9fccb, user_id=ff544aeee2e14b32807ccfbdd03f8fb9, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:44.191706 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0c17f-7de5-4248-b7a4-cb2fc5c8ee24 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:44.208729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1194/2394] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:18:44 2022] PUT /identity/v3/projects/4cb7628373d148baa468e2cabfd9fccb/users/ff544aeee2e14b32807ccfbdd03f8fb9/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:18:50.229244 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367da8a9-9b06-4a78-bbde-67370b4d840f None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:50.229802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367da8a9-9b06-4a78-bbde-67370b4d840f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:50.229802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367da8a9-9b06-4a78-bbde-67370b4d840f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:50.273267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-367da8a9-9b06-4a78-bbde-67370b4d840f None None] MFA Rules not processed for user `ff544aeee2e14b32807ccfbdd03f8fb9`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:18:50.274837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-367da8a9-9b06-4a78-bbde-67370b4d840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:50.361021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:18:50.367839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1201/2395] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:18:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:18:50.382476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0c17f-7de5-4248-b7a4-cb2fc5c8ee24 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:18:50.385762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d4d0c17f-7de5-4248-b7a4-cb2fc5c8ee24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:18:50.484065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0c17f-7de5-4248-b7a4-cb2fc5c8ee24 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:18:50.492338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8eaa8-0b04-4485-b096-7e9eed0cea5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:18:50.493226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8eaa8-0b04-4485-b096-7e9eed0cea5a service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:18:50.493226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8eaa8-0b04-4485-b096-7e9eed0cea5a service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:18:50.493476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8eaa8-0b04-4485-b096-7e9eed0cea5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:18:50.531312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:18:50.569651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8eaa8-0b04-4485-b096-7e9eed0cea5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:18:50.571397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8eaa8-0b04-4485-b096-7e9eed0cea5a service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:18:50.585686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1195/2396] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:18:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:19:00.434917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:19:00.606178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=16 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:19:12.874551 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367da8a9-9b06-4a78-bbde-67370b4d840f None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:19:12.874551 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-367da8a9-9b06-4a78-bbde-67370b4d840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:19:12.930293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367da8a9-9b06-4a78-bbde-67370b4d840f None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:19:12.933910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee33a2a-4329-480b-bb58-2a65a0b1d597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:19:12.934663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee33a2a-4329-480b-bb58-2a65a0b1d597 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:19:12.934936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee33a2a-4329-480b-bb58-2a65a0b1d597 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:19:12.935201 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee33a2a-4329-480b-bb58-2a65a0b1d597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:19:12.941914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8ee33a2a-4329-480b-bb58-2a65a0b1d597 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:19:13.021366 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee33a2a-4329-480b-bb58-2a65a0b1d597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:19:13.022800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee33a2a-4329-480b-bb58-2a65a0b1d597 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:19:13.043214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1202/2397] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:19:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:19:23.040566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=61 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:20:09.900341 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8eaa8-0b04-4485-b096-7e9eed0cea5a service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:20:09.902040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2af8eaa8-0b04-4485-b096-7e9eed0cea5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:20:09.996380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8eaa8-0b04-4485-b096-7e9eed0cea5a service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:20:10.001363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb920d-d377-4a53-b50a-f5c9ae02657c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:20:10.001946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb920d-d377-4a53-b50a-f5c9ae02657c service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:20:10.002245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb920d-d377-4a53-b50a-f5c9ae02657c service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:20:10.002510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb920d-d377-4a53-b50a-f5c9ae02657c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:20:10.004186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e9cb920d-d377-4a53-b50a-f5c9ae02657c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:20:10.015860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb920d-d377-4a53-b50a-f5c9ae02657c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:20:10.017515 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb920d-d377-4a53-b50a-f5c9ae02657c service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:20:10.027943 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1196/2398] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:20:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:20:20.023671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=43 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:21:14.206900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee33a2a-4329-480b-bb58-2a65a0b1d597 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:21:14.226163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee33a2a-4329-480b-bb58-2a65a0b1d597 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:21:14.230373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717e3f3-7e9f-41bf-be59-af8c5e727af2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:21:14.230961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717e3f3-7e9f-41bf-be59-af8c5e727af2 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:21:14.231235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717e3f3-7e9f-41bf-be59-af8c5e727af2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:21:14.231499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717e3f3-7e9f-41bf-be59-af8c5e727af2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:21:14.233219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f717e3f3-7e9f-41bf-be59-af8c5e727af2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:21:14.306296 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717e3f3-7e9f-41bf-be59-af8c5e727af2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:21:14.307727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717e3f3-7e9f-41bf-be59-af8c5e727af2 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:21:14.316779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1203/2399] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:21:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:21:14.712229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb920d-d377-4a53-b50a-f5c9ae02657c service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:21:14.731797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb920d-d377-4a53-b50a-f5c9ae02657c service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:21:14.735454 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64875ed0-9494-4dc7-ae8b-4edb1df58109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:21:14.736044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64875ed0-9494-4dc7-ae8b-4edb1df58109 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:21:14.736320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64875ed0-9494-4dc7-ae8b-4edb1df58109 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:21:14.736590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64875ed0-9494-4dc7-ae8b-4edb1df58109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:21:14.738449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-64875ed0-9494-4dc7-ae8b-4edb1df58109 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:21:14.810184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64875ed0-9494-4dc7-ae8b-4edb1df58109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:21:14.811620 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64875ed0-9494-4dc7-ae8b-4edb1df58109 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:21:14.820571 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1197/2400] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:21:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:21:24.312599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=39 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:21:24.816373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=38 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:21:55.308497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717e3f3-7e9f-41bf-be59-af8c5e727af2 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:21:55.321452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717e3f3-7e9f-41bf-be59-af8c5e727af2 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:21:55.327096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c2905-0dfd-4763-bb27-099847dfe2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:21:55.327691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c2905-0dfd-4763-bb27-099847dfe2bc service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:21:55.327963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c2905-0dfd-4763-bb27-099847dfe2bc service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:21:55.328226 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c2905-0dfd-4763-bb27-099847dfe2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:21:55.329933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9b5c2905-0dfd-4763-bb27-099847dfe2bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:21:55.388143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c2905-0dfd-4763-bb27-099847dfe2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:21:55.389715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c2905-0dfd-4763-bb27-099847dfe2bc service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:21:55.400561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1204/2401] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:21:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:22:05.395510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=28 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:22:42.123499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5b333-3e6b-4ae1-a2b1-57c8ca371e10 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:22:42.124068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5b333-3e6b-4ae1-a2b1-57c8ca371e10 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:22:42.124068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5b333-3e6b-4ae1-a2b1-57c8ca371e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:22:42.152346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c0b5b333-3e6b-4ae1-a2b1-57c8ca371e10 None None] MFA Rules not processed for user `ccad0b7ad5274d02b43f9c2fb857452a`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:22:42.153577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c0b5b333-3e6b-4ae1-a2b1-57c8ca371e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:22:42.160793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1198/2402] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:22:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:22:42.468640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c2905-0dfd-4763-bb27-099847dfe2bc service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:22:42.470154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9b5c2905-0dfd-4763-bb27-099847dfe2bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:22:42.530122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c2905-0dfd-4763-bb27-099847dfe2bc service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:22:42.533766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166829c4-8784-4bb8-a217-49764a304c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:22:42.534395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166829c4-8784-4bb8-a217-49764a304c6e service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:22:42.534672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166829c4-8784-4bb8-a217-49764a304c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:22:42.534939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166829c4-8784-4bb8-a217-49764a304c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:22:42.536661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-166829c4-8784-4bb8-a217-49764a304c6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:22:42.546555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166829c4-8784-4bb8-a217-49764a304c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:22:42.547958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166829c4-8784-4bb8-a217-49764a304c6e service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:22:42.556589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1205/2403] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:22:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:22:42.945829 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5b333-3e6b-4ae1-a2b1-57c8ca371e10 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:22:42.947227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c0b5b333-3e6b-4ae1-a2b1-57c8ca371e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:22:42.962253 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5b333-3e6b-4ae1-a2b1-57c8ca371e10 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:22:42.967165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5565677-3914-41d6-a165-643bdc3013b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:22:42.967760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5565677-3914-41d6-a165-643bdc3013b6 service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:22:42.968035 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5565677-3914-41d6-a165-643bdc3013b6 service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:22:42.968303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5565677-3914-41d6-a165-643bdc3013b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:22:42.970060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-f5565677-3914-41d6-a165-643bdc3013b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:22:42.984770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5565677-3914-41d6-a165-643bdc3013b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:22:42.986747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5565677-3914-41d6-a165-643bdc3013b6 service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:22:42.997521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1199/2404] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:22:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:22:43.485143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166829c4-8784-4bb8-a217-49764a304c6e service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:22:43.503792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166829c4-8784-4bb8-a217-49764a304c6e service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:22:43.507435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036e117-11c1-40ff-98fe-72057828b842 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:22:43.508058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036e117-11c1-40ff-98fe-72057828b842 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:22:43.510768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036e117-11c1-40ff-98fe-72057828b842 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:22:43.510768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036e117-11c1-40ff-98fe-72057828b842 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:22:43.510768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8036e117-11c1-40ff-98fe-72057828b842 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:22:43.571768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036e117-11c1-40ff-98fe-72057828b842 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:22:43.573081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036e117-11c1-40ff-98fe-72057828b842 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:22:43.582041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1206/2405] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:22:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:22:47.074906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5565677-3914-41d6-a165-643bdc3013b6 service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:22:47.088559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5565677-3914-41d6-a165-643bdc3013b6 service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:22:47.095128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317f4f8-b1ac-4d81-85c5-0b3771eefd1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:22:47.095727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317f4f8-b1ac-4d81-85c5-0b3771eefd1f service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:22:47.096025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317f4f8-b1ac-4d81-85c5-0b3771eefd1f service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:22:47.096289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317f4f8-b1ac-4d81-85c5-0b3771eefd1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:22:47.098062 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7317f4f8-b1ac-4d81-85c5-0b3771eefd1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:22:47.110185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317f4f8-b1ac-4d81-85c5-0b3771eefd1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:22:47.111745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317f4f8-b1ac-4d81-85c5-0b3771eefd1f service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:22:47.123003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1200/2406] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:22:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:22:53.577892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=58 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:22:54.245195 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036e117-11c1-40ff-98fe-72057828b842 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:22:54.275613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036e117-11c1-40ff-98fe-72057828b842 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:22:54.282081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062f4c7-8b54-4361-9669-a976984e8e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:22:54.282712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062f4c7-8b54-4361-9669-a976984e8e02 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:22:54.283010 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062f4c7-8b54-4361-9669-a976984e8e02 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:22:54.283283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062f4c7-8b54-4361-9669-a976984e8e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:22:54.285026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a062f4c7-8b54-4361-9669-a976984e8e02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:22:54.297535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062f4c7-8b54-4361-9669-a976984e8e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:22:54.299163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062f4c7-8b54-4361-9669-a976984e8e02 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:22:54.309667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1207/2407] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:22:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:22:56.071855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317f4f8-b1ac-4d81-85c5-0b3771eefd1f service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:22:56.091277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317f4f8-b1ac-4d81-85c5-0b3771eefd1f service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:22:56.094915 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037810b-8fa5-4010-930e-9c8f25f82b7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:22:56.098183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037810b-8fa5-4010-930e-9c8f25f82b7a service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:22:56.098183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037810b-8fa5-4010-930e-9c8f25f82b7a service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:22:56.098183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037810b-8fa5-4010-930e-9c8f25f82b7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:22:56.098183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2037810b-8fa5-4010-930e-9c8f25f82b7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:22:56.112039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037810b-8fa5-4010-930e-9c8f25f82b7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:22:56.113493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2037810b-8fa5-4010-930e-9c8f25f82b7a service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:22:56.125345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1201/2408] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:22:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:22:56.683990 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062f4c7-8b54-4361-9669-a976984e8e02 service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:22:56.692962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062f4c7-8b54-4361-9669-a976984e8e02 service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:22:56.697213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066450e-4692-43ab-8747-bf71aed0aa9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:22:56.701218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066450e-4692-43ab-8747-bf71aed0aa9c service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:22:56.701218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066450e-4692-43ab-8747-bf71aed0aa9c service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:22:56.701218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066450e-4692-43ab-8747-bf71aed0aa9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:22:56.701218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2066450e-4692-43ab-8747-bf71aed0aa9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:22:56.710337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066450e-4692-43ab-8747-bf71aed0aa9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:22:56.712014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066450e-4692-43ab-8747-bf71aed0aa9c service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:22:56.720008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1208/2409] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:22:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:23:06.120493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=28 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:23:06.716422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=19 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:25:14.857811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037810b-8fa5-4010-930e-9c8f25f82b7a service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:25:14.869070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037810b-8fa5-4010-930e-9c8f25f82b7a service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:25:14.874641 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482141-c9c6-4d70-9894-dd175c63ddff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:25:14.875221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482141-c9c6-4d70-9894-dd175c63ddff service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:25:14.875497 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482141-c9c6-4d70-9894-dd175c63ddff service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:25:14.875765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a482141-c9c6-4d70-9894-dd175c63ddff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:25:14.884339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a482141-c9c6-4d70-9894-dd175c63ddff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:25:14.885892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a482141-c9c6-4d70-9894-dd175c63ddff service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:25:14.896901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1202/2410] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:25:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:25:15.069764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066450e-4692-43ab-8747-bf71aed0aa9c service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:25:15.079659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066450e-4692-43ab-8747-bf71aed0aa9c service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:25:15.083884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310c3c5-a123-4619-a2d4-ddca9292d9ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:25:15.084509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310c3c5-a123-4619-a2d4-ddca9292d9ad service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:25:15.084785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310c3c5-a123-4619-a2d4-ddca9292d9ad service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:25:15.085106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310c3c5-a123-4619-a2d4-ddca9292d9ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:25:15.086970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8310c3c5-a123-4619-a2d4-ddca9292d9ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:25:15.164061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310c3c5-a123-4619-a2d4-ddca9292d9ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:25:15.165474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310c3c5-a123-4619-a2d4-ddca9292d9ad service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:25:15.176111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1209/2411] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:25:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:25:15.198490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482141-c9c6-4d70-9894-dd175c63ddff service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:25:15.207983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a482141-c9c6-4d70-9894-dd175c63ddff service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:25:15.211964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad78c2-b134-4f38-ace2-8edb5ffd543b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:25:15.212580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad78c2-b134-4f38-ace2-8edb5ffd543b service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:25:15.212881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad78c2-b134-4f38-ace2-8edb5ffd543b service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:25:15.213151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad78c2-b134-4f38-ace2-8edb5ffd543b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:25:15.221914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad78c2-b134-4f38-ace2-8edb5ffd543b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:25:15.223764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad78c2-b134-4f38-ace2-8edb5ffd543b service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:25:15.233058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1203/2412] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:25:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:25:15.316627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310c3c5-a123-4619-a2d4-ddca9292d9ad service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:25:15.325437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310c3c5-a123-4619-a2d4-ddca9292d9ad service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:25:15.329401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e3914-c780-4132-9b5a-d49b89064ec4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:25:15.330015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e3914-c780-4132-9b5a-d49b89064ec4 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:25:15.330297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e3914-c780-4132-9b5a-d49b89064ec4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:25:15.330562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e3914-c780-4132-9b5a-d49b89064ec4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:25:15.397438 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e3914-c780-4132-9b5a-d49b89064ec4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:25:15.398837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e3914-c780-4132-9b5a-d49b89064ec4 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:25:15.417729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1210/2413] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:25:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:25:15.464286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad78c2-b134-4f38-ace2-8edb5ffd543b service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:25:15.474451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad78c2-b134-4f38-ace2-8edb5ffd543b service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:25:15.478051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6856533-45c8-4df6-bea0-0952e338c36f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:25:15.478649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6856533-45c8-4df6-bea0-0952e338c36f service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:25:15.478904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6856533-45c8-4df6-bea0-0952e338c36f service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:25:15.479170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6856533-45c8-4df6-bea0-0952e338c36f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:25:15.489215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6856533-45c8-4df6-bea0-0952e338c36f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:25:15.490694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6856533-45c8-4df6-bea0-0952e338c36f service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:25:15.499974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1204/2414] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:25:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:25:25.415015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=70 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:25:25.495616 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=18 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:26:54.860523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e3914-c780-4132-9b5a-d49b89064ec4 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:26:54.869797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e3914-c780-4132-9b5a-d49b89064ec4 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:26:54.873700 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6540dc-4e2c-4282-b001-872f9d6cb8d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:26:54.874327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6540dc-4e2c-4282-b001-872f9d6cb8d6 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:26:54.874605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6540dc-4e2c-4282-b001-872f9d6cb8d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:26:54.874871 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6540dc-4e2c-4282-b001-872f9d6cb8d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:26:54.883786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6540dc-4e2c-4282-b001-872f9d6cb8d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:26:54.885524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6540dc-4e2c-4282-b001-872f9d6cb8d6 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:26:54.896428 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1211/2415] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:26:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:26:55.105680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6856533-45c8-4df6-bea0-0952e338c36f service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:26:55.116384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765a3de-5ffc-4619-bd0a-95b238de8a08 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:26:55.116384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765a3de-5ffc-4619-bd0a-95b238de8a08 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:26:55.116384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765a3de-5ffc-4619-bd0a-95b238de8a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:26:55.118822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6856533-45c8-4df6-bea0-0952e338c36f service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:26:55.129424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725026c-93bc-4d0e-b90d-e3a54e5a40c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:26:55.129424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725026c-93bc-4d0e-b90d-e3a54e5a40c1 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:26:55.129424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725026c-93bc-4d0e-b90d-e3a54e5a40c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:26:55.129424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725026c-93bc-4d0e-b90d-e3a54e5a40c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:26:55.147088 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725026c-93bc-4d0e-b90d-e3a54e5a40c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:26:55.148536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725026c-93bc-4d0e-b90d-e3a54e5a40c1 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:26:55.150664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-5765a3de-5ffc-4619-bd0a-95b238de8a08 None None] MFA Rules not processed for user `f493a8495dd64a94b12e159862167408`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:26:55.157375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5765a3de-5ffc-4619-bd0a-95b238de8a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:26:55.160640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1212/2416] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:26:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:26:55.161557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765a3de-5ffc-4619-bd0a-95b238de8a08 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:26:55.175032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765a3de-5ffc-4619-bd0a-95b238de8a08 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:26:55.179140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791887d-aadf-450e-b46e-b74d220fd3ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:26:55.179748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791887d-aadf-450e-b46e-b74d220fd3ac service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:26:55.180021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791887d-aadf-450e-b46e-b74d220fd3ac service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:26:55.180328 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791887d-aadf-450e-b46e-b74d220fd3ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:26:55.187214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1205/2417] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:26:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:26:55.187518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9791887d-aadf-450e-b46e-b74d220fd3ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:26:55.258749 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791887d-aadf-450e-b46e-b74d220fd3ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:26:55.260213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791887d-aadf-450e-b46e-b74d220fd3ac service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:26:55.271149 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1213/2418] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:26:55.914188 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725026c-93bc-4d0e-b90d-e3a54e5a40c1 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:26:55.923127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725026c-93bc-4d0e-b90d-e3a54e5a40c1 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:26:55.927542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637d750-6527-4fb1-9c00-8d4d618e9c72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:26:55.928135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637d750-6527-4fb1-9c00-8d4d618e9c72 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:26:55.928410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637d750-6527-4fb1-9c00-8d4d618e9c72 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:26:55.928678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637d750-6527-4fb1-9c00-8d4d618e9c72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:26:55.938714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637d750-6527-4fb1-9c00-8d4d618e9c72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:26:55.940165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c637d750-6527-4fb1-9c00-8d4d618e9c72 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:26:55.972288 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1206/2419] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:26:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:26:56.022754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791887d-aadf-450e-b46e-b74d220fd3ac service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:26:56.024222 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9791887d-aadf-450e-b46e-b74d220fd3ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:26:56.160335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791887d-aadf-450e-b46e-b74d220fd3ac service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:26:56.164435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac92b6-2610-4cb5-8db8-feb8134a907a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33003a9bf4454f5bac2e6ad73544efc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:26:56.165055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac92b6-2610-4cb5-8db8-feb8134a907a service glance] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:26:56.165653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac92b6-2610-4cb5-8db8-feb8134a907a service glance] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:26:56.165653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac92b6-2610-4cb5-8db8-feb8134a907a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:26:56.167637 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a8ac92b6-2610-4cb5-8db8-feb8134a907a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application [None req-a8ac92b6-2610-4cb5-8db8-feb8134a907a service glance] Could not find project: cb9a129996ad4cbd9e5e14b7dfac224e.: keystone.exception.ProjectNotFound: Could not find project: cb9a129996ad4cbd9e5e14b7dfac224e. Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Aug 09 17:26:56.183498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:26:56.184557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self._validate_token_resources() Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 09 17:26:56.185648 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.get_or_create( Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application with Lock( Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._enter() Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.creator() Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application created_value = creator( Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 09 17:26:56.187532 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 09 17:26:56.188499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: cb9a129996ad4cbd9e5e14b7dfac224e. Aug 09 17:26:56.188499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: ERROR keystone.server.flask.application Aug 09 17:26:56.188499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1214/2420] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:26:56 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Aug 09 17:26:58.458699 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637d750-6527-4fb1-9c00-8d4d618e9c72 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:26:58.460232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c637d750-6527-4fb1-9c00-8d4d618e9c72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:26:58.504757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637d750-6527-4fb1-9c00-8d4d618e9c72 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:26:58.516909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:26:58.518716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:26:58.518994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:26:58.519266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] PATH_INFO: `/v3/users/6c5f09d5c2864bb6b5251018e05c31ed` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:26:58.576539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c5f09d5c2864bb6b5251018e05c31ed)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:26:58.579436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:26:58.903865 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c5f09d5c2864bb6b5251018e05c31ed'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:26:59.016950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c5f09d5c2864bb6b5251018e05c31ed'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:26:59.025323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c5f09d5c2864bb6b5251018e05c31ed'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:26:59.039167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c5f09d5c2864bb6b5251018e05c31ed'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:26:59.039872 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c5f09d5c2864bb6b5251018e05c31ed'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:26:59.046615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1207/2421] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:26:58 2022] DELETE /identity/v3/users/6c5f09d5c2864bb6b5251018e05c31ed => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:26:59.051151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda128d-8c93-4c31-9c14-707346bc9768 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:26:59.051622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda128d-8c93-4c31-9c14-707346bc9768 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:26:59.051752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda128d-8c93-4c31-9c14-707346bc9768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:26:59.086880 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-bdda128d-8c93-4c31-9c14-707346bc9768 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:26:59.088136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bdda128d-8c93-4c31-9c14-707346bc9768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:26:59.145924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:26:59.160906 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1215/2422] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:26:59.172769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:26:59.174307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:26:59.280259 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84d44a-c960-445a-b64e-765fe92bab0e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:26:59.292302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92bdd7-9ba8-4c2e-a8ab-4eb5c7c0b296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:26:59.292927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92bdd7-9ba8-4c2e-a8ab-4eb5c7c0b296 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:26:59.293200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92bdd7-9ba8-4c2e-a8ab-4eb5c7c0b296 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:26:59.293468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92bdd7-9ba8-4c2e-a8ab-4eb5c7c0b296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:26:59.302467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:26:59.304648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92bdd7-9ba8-4c2e-a8ab-4eb5c7c0b296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:26:59.306177 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92bdd7-9ba8-4c2e-a8ab-4eb5c7c0b296 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:26:59.318924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1208/2423] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:26:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:00.413627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda128d-8c93-4c31-9c14-707346bc9768 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:00.415148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bdda128d-8c93-4c31-9c14-707346bc9768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:00.435411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdda128d-8c93-4c31-9c14-707346bc9768 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:00.451792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:00.452451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:00.455523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:00.455523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] PATH_INFO: `/v3/projects/cb0622e26cc5416c8a507c20c88f6cc6` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:00.455523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb0622e26cc5416c8a507c20c88f6cc6)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:00.456217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:00.640908 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0622e26cc5416c8a507c20c88f6cc6', 'request_id': 'req-4c655276-ef36-4ed2-bc8a-acf4ec75b236'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:00.687521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92bdd7-9ba8-4c2e-a8ab-4eb5c7c0b296 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:00.688982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5f92bdd7-9ba8-4c2e-a8ab-4eb5c7c0b296 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:00.731454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0622e26cc5416c8a507c20c88f6cc6', 'request_id': 'req-4c655276-ef36-4ed2-bc8a-acf4ec75b236'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:00.742087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0622e26cc5416c8a507c20c88f6cc6', 'request_id': 'req-4c655276-ef36-4ed2-bc8a-acf4ec75b236'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:00.755307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] The token cache is being invalidate because project cb0622e26cc5416c8a507c20c88f6cc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:00.755686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:00.757494 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:00.759772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1216/2424] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:00 2022] DELETE /identity/v3/projects/cb0622e26cc5416c8a507c20c88f6cc6 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:00.767554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:00.770410 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:00.861941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92bdd7-9ba8-4c2e-a8ab-4eb5c7c0b296 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:00.947613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c655276-ef36-4ed2-bc8a-acf4ec75b236 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:00.958249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13270f-c3d4-45a8-9f45-86d8898f8e56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:00.958867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13270f-c3d4-45a8-9f45-86d8898f8e56 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:00.959141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13270f-c3d4-45a8-9f45-86d8898f8e56 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:00.960546 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:00.962078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13270f-c3d4-45a8-9f45-86d8898f8e56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:00.962194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:00.962390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:00.962650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] PATH_INFO: `/v3/users/ec1225ec397f43cb821e4516dacbc6e0` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:00.965321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4f13270f-c3d4-45a8-9f45-86d8898f8e56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:00.982586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec1225ec397f43cb821e4516dacbc6e0)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:00.984761 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:00.985118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13270f-c3d4-45a8-9f45-86d8898f8e56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:00.993217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13270f-c3d4-45a8-9f45-86d8898f8e56 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:01.008987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1209/2425] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:01.010194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13270f-c3d4-45a8-9f45-86d8898f8e56 service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:01.011881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4f13270f-c3d4-45a8-9f45-86d8898f8e56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:01.125719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:01.147320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13270f-c3d4-45a8-9f45-86d8898f8e56 service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:01.167539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d30efe-b12a-45e6-9aa3-46ecac340e25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:01.167539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d30efe-b12a-45e6-9aa3-46ecac340e25 service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:01.167539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d30efe-b12a-45e6-9aa3-46ecac340e25 service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:01.167539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d30efe-b12a-45e6-9aa3-46ecac340e25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:01.167895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c4d30efe-b12a-45e6-9aa3-46ecac340e25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:01.177421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d30efe-b12a-45e6-9aa3-46ecac340e25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:01.180507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d30efe-b12a-45e6-9aa3-46ecac340e25 service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:01.194602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1210/2426] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:01.215530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d30efe-b12a-45e6-9aa3-46ecac340e25 service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:01.220177 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c4d30efe-b12a-45e6-9aa3-46ecac340e25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:01.267653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec1225ec397f43cb821e4516dacbc6e0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:01.269770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d30efe-b12a-45e6-9aa3-46ecac340e25 service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:01.296679 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:01.298524 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:01.299800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:01.302307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] PATH_INFO: `/v3/users/84c46c4817184c9d8554c956bb7e48dc` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:01.306608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c46c4817184c9d8554c956bb7e48dc)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:01.314882 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:01.351831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec1225ec397f43cb821e4516dacbc6e0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:01.387400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:01.394080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec1225ec397f43cb821e4516dacbc6e0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:01.395194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec1225ec397f43cb821e4516dacbc6e0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:01.395860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec1225ec397f43cb821e4516dacbc6e0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:01.426865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1217/2427] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:27:00 2022] DELETE /identity/v3/users/ec1225ec397f43cb821e4516dacbc6e0 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:01.653719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c46c4817184c9d8554c956bb7e48dc'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:01.739040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c46c4817184c9d8554c956bb7e48dc'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:01.747221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c46c4817184c9d8554c956bb7e48dc'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:01.760450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c46c4817184c9d8554c956bb7e48dc'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:01.761222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c46c4817184c9d8554c956bb7e48dc'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:01.769998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1211/2428] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:27:01 2022] DELETE /identity/v3/users/84c46c4817184c9d8554c956bb7e48dc => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:02.129179 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:02.131264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:02.244505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ff50c-aa51-48ca-a0e1-c92609cf2ff7 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:02.359242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:02.359885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:02.360163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:02.360434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] PATH_INFO: `/v3/projects/fdeb9e648a4447d1b9ce7e035ca3be2e` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:02.361713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdeb9e648a4447d1b9ce7e035ca3be2e)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:02.363679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:02.441169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:02.443529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:02.459599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7187a86-baa4-4726-82ee-0e721a2c37ec admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:02.469283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:02.470380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:02.470848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:02.471276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] PATH_INFO: `/v3/projects/41404ea74a4c4acc894511223e5ed2a9` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:02.500404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41404ea74a4c4acc894511223e5ed2a9)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:02.512179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:02.597016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdeb9e648a4447d1b9ce7e035ca3be2e', 'request_id': 'req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:02.676276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41404ea74a4c4acc894511223e5ed2a9', 'request_id': 'req-98de0435-fe71-4333-9320-d38073fb2dc8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:02.741423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdeb9e648a4447d1b9ce7e035ca3be2e', 'request_id': 'req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:02.749945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdeb9e648a4447d1b9ce7e035ca3be2e', 'request_id': 'req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:02.761012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] The token cache is being invalidate because project fdeb9e648a4447d1b9ce7e035ca3be2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:02.761341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:02.762027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ab2f6b11-8ea9-4ce1-b3e0-480e72e225fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:02.763507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1218/2429] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:02 2022] DELETE /identity/v3/projects/fdeb9e648a4447d1b9ce7e035ca3be2e => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:02.770642 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643d7a5-f8cc-418c-8f37-c949015e54ff None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:02.770916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643d7a5-f8cc-418c-8f37-c949015e54ff None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:02.771180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643d7a5-f8cc-418c-8f37-c949015e54ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:02.807276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6643d7a5-f8cc-418c-8f37-c949015e54ff None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:02.808588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6643d7a5-f8cc-418c-8f37-c949015e54ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:02.828398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:02.881078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41404ea74a4c4acc894511223e5ed2a9', 'request_id': 'req-98de0435-fe71-4333-9320-d38073fb2dc8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:02.890601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41404ea74a4c4acc894511223e5ed2a9', 'request_id': 'req-98de0435-fe71-4333-9320-d38073fb2dc8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:02.907586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1219/2430] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:27:02.909186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91aeeff-7a03-4581-a4f3-5dd153977dac None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:02.909568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91aeeff-7a03-4581-a4f3-5dd153977dac None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:02.909852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91aeeff-7a03-4581-a4f3-5dd153977dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:02.910177 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] The token cache is being invalidate because project 41404ea74a4c4acc894511223e5ed2a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:02.910471 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:02.911211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-98de0435-fe71-4333-9320-d38073fb2dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:02.912928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1212/2431] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:02 2022] DELETE /identity/v3/projects/41404ea74a4c4acc894511223e5ed2a9 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:27:02.917433 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e000ef-a594-4caa-bfe3-f61f70660234 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:02.917720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e000ef-a594-4caa-bfe3-f61f70660234 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:02.918086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e000ef-a594-4caa-bfe3-f61f70660234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:02.933033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d91aeeff-7a03-4581-a4f3-5dd153977dac None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:02.934376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d91aeeff-7a03-4581-a4f3-5dd153977dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:02.942526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-10e000ef-a594-4caa-bfe3-f61f70660234 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:02.943792 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-10e000ef-a594-4caa-bfe3-f61f70660234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:02.944440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1220/2432] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:27:02.949055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91aeeff-7a03-4581-a4f3-5dd153977dac None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:02.958736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91aeeff-7a03-4581-a4f3-5dd153977dac None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:02.963623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f491470-1d54-4454-8e46-fa44ff7c4cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:02.964499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491470-1d54-4454-8e46-fa44ff7c4cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:02.964612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491470-1d54-4454-8e46-fa44ff7c4cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:02.974501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1213/2433] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:27:02.974643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491470-1d54-4454-8e46-fa44ff7c4cd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:02.974643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f491470-1d54-4454-8e46-fa44ff7c4cd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:27:02.974643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f491470-1d54-4454-8e46-fa44ff7c4cd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:02.974839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe095fd-334a-4d94-8828-ae6b7e9f513b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:02.974839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe095fd-334a-4d94-8828-ae6b7e9f513b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:02.974839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe095fd-334a-4d94-8828-ae6b7e9f513b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:02.975061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f491470-1d54-4454-8e46-fa44ff7c4cd8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:02.984592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1221/2434] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:27:02.985711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f491470-1d54-4454-8e46-fa44ff7c4cd8 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:02.993303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-3fe095fd-334a-4d94-8828-ae6b7e9f513b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:02.995413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3fe095fd-334a-4d94-8828-ae6b7e9f513b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:02.995998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f491470-1d54-4454-8e46-fa44ff7c4cd8 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.002835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e7487-fd53-4bb2-871f-331ba92d2cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.010394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e7487-fd53-4bb2-871f-331ba92d2cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.010394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e7487-fd53-4bb2-871f-331ba92d2cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.010394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e7487-fd53-4bb2-871f-331ba92d2cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.010394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e7487-fd53-4bb2-871f-331ba92d2cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.010394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e7487-fd53-4bb2-871f-331ba92d2cf0 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.011153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1214/2435] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:03.012354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe095fd-334a-4d94-8828-ae6b7e9f513b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.023213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe095fd-334a-4d94-8828-ae6b7e9f513b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.028363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7c4c4-f26e-47d2-8285-69dfd1592b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.029002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7c4c4-f26e-47d2-8285-69dfd1592b71 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.029284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7c4c4-f26e-47d2-8285-69dfd1592b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.029553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada7c4c4-f26e-47d2-8285-69dfd1592b71 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.030373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7c4c4-f26e-47d2-8285-69dfd1592b71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:27:03.030765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7c4c4-f26e-47d2-8285-69dfd1592b71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.032411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada7c4c4-f26e-47d2-8285-69dfd1592b71 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.044692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1215/2436] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:27:03.049470 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7c4c4-f26e-47d2-8285-69dfd1592b71 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.059903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada7c4c4-f26e-47d2-8285-69dfd1592b71 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.066167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f02d3a-0bed-4b63-992f-4b1338acd04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.070668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f02d3a-0bed-4b63-992f-4b1338acd04e admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.070668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f02d3a-0bed-4b63-992f-4b1338acd04e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.070668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f02d3a-0bed-4b63-992f-4b1338acd04e admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.070668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f02d3a-0bed-4b63-992f-4b1338acd04e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.071643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f02d3a-0bed-4b63-992f-4b1338acd04e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.077326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:03.100275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1222/2437] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:02 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:03.104265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e7487-fd53-4bb2-871f-331ba92d2cf0 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.190942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e7487-fd53-4bb2-871f-331ba92d2cf0 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.196318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d16dcd-a83e-42c7-a4a7-1953aba387a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.196930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d16dcd-a83e-42c7-a4a7-1953aba387a9 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.197209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d16dcd-a83e-42c7-a4a7-1953aba387a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.197478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d16dcd-a83e-42c7-a4a7-1953aba387a9 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.198132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d16dcd-a83e-42c7-a4a7-1953aba387a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.200748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d16dcd-a83e-42c7-a4a7-1953aba387a9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.210006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1216/2438] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:03 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:27:03.215759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f02d3a-0bed-4b63-992f-4b1338acd04e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.309245 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f02d3a-0bed-4b63-992f-4b1338acd04e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.322251 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1223/2439] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:27:03 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:03.322902 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881538ef-a8f1-4965-97ac-b112a9ef31cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.323631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881538ef-a8f1-4965-97ac-b112a9ef31cd admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.324017 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881538ef-a8f1-4965-97ac-b112a9ef31cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.324295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881538ef-a8f1-4965-97ac-b112a9ef31cd admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.325099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881538ef-a8f1-4965-97ac-b112a9ef31cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.326186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d16dcd-a83e-42c7-a4a7-1953aba387a9 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.329391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881538ef-a8f1-4965-97ac-b112a9ef31cd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.339123 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d16dcd-a83e-42c7-a4a7-1953aba387a9 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.346401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4dbbf-782f-4f44-b524-7fed2dea6897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.346959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4dbbf-782f-4f44-b524-7fed2dea6897 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.347562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4dbbf-782f-4f44-b524-7fed2dea6897 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.347913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4dbbf-782f-4f44-b524-7fed2dea6897 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.348639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4dbbf-782f-4f44-b524-7fed2dea6897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.350685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4dbbf-782f-4f44-b524-7fed2dea6897 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.363077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1224/2440] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:27:03.368361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4dbbf-782f-4f44-b524-7fed2dea6897 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.380848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4dbbf-782f-4f44-b524-7fed2dea6897 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.387607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70189fa4-2044-40ac-ba5f-a82c175d8f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.388270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70189fa4-2044-40ac-ba5f-a82c175d8f47 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.388560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70189fa4-2044-40ac-ba5f-a82c175d8f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.388933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70189fa4-2044-40ac-ba5f-a82c175d8f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.390169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70189fa4-2044-40ac-ba5f-a82c175d8f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.393979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70189fa4-2044-40ac-ba5f-a82c175d8f47 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.404153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1225/2441] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:27:03.409059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70189fa4-2044-40ac-ba5f-a82c175d8f47 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.421854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70189fa4-2044-40ac-ba5f-a82c175d8f47 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.427937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d08c46-91d4-4e4a-8572-518bce4fa21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.428681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d08c46-91d4-4e4a-8572-518bce4fa21a admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.428905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d08c46-91d4-4e4a-8572-518bce4fa21a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.429205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d08c46-91d4-4e4a-8572-518bce4fa21a admin admin] PATH_INFO: `/v3/projects/87c6e97feec34b228c2419572723f5a3/users/ff2651d4ed9b497fa8aa037cc4dc4ea3/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.448963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d08c46-91d4-4e4a-8572-518bce4fa21a admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c6e97feec34b228c2419572723f5a3, user_id=ff2651d4ed9b497fa8aa037cc4dc4ea3, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.452647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d08c46-91d4-4e4a-8572-518bce4fa21a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.465941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1217/2442] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:27:03 2022] POST /identity/v3/users => generated 291 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:27:03.469722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881538ef-a8f1-4965-97ac-b112a9ef31cd admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.483582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881538ef-a8f1-4965-97ac-b112a9ef31cd admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.492402 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d523aac-85df-4562-a593-7689fa438c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.493150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d523aac-85df-4562-a593-7689fa438c6f admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.493534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d523aac-85df-4562-a593-7689fa438c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.493997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d523aac-85df-4562-a593-7689fa438c6f admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.494751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d523aac-85df-4562-a593-7689fa438c6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.497211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d523aac-85df-4562-a593-7689fa438c6f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.524625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1218/2443] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:27:03.526350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d523aac-85df-4562-a593-7689fa438c6f admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.538935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d523aac-85df-4562-a593-7689fa438c6f admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.544191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec221f-06b8-455d-b62a-aea9cc5a5812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.547660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec221f-06b8-455d-b62a-aea9cc5a5812 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.547660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec221f-06b8-455d-b62a-aea9cc5a5812 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.547660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ec221f-06b8-455d-b62a-aea9cc5a5812 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.547660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec221f-06b8-455d-b62a-aea9cc5a5812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.548268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ec221f-06b8-455d-b62a-aea9cc5a5812 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.559528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1219/2444] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:27:03.564539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec221f-06b8-455d-b62a-aea9cc5a5812 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.575799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ec221f-06b8-455d-b62a-aea9cc5a5812 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.580549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53081c-afa7-48f0-874e-b892a4f576b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.581190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53081c-afa7-48f0-874e-b892a4f576b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.581467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53081c-afa7-48f0-874e-b892a4f576b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.581734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53081c-afa7-48f0-874e-b892a4f576b4 admin admin] PATH_INFO: `/v3/projects/58c44a80e3c449749623917923ecd611/users/b51405380a1e4f369f86ee6d187caff9/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.597897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53081c-afa7-48f0-874e-b892a4f576b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c44a80e3c449749623917923ecd611, user_id=b51405380a1e4f369f86ee6d187caff9, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.601856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c53081c-afa7-48f0-874e-b892a4f576b4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.602180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1226/2445] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:03 2022] PUT /identity/v3/projects/87c6e97feec34b228c2419572723f5a3/users/ff2651d4ed9b497fa8aa037cc4dc4ea3/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:27:03.613101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7840a4-e0c5-4bac-a125-eb6aa2c0e2a7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.613749 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7840a4-e0c5-4bac-a125-eb6aa2c0e2a7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.614152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7840a4-e0c5-4bac-a125-eb6aa2c0e2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.647306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-1f7840a4-e0c5-4bac-a125-eb6aa2c0e2a7 None None] MFA Rules not processed for user `ff2651d4ed9b497fa8aa037cc4dc4ea3`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:03.648562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-1f7840a4-e0c5-4bac-a125-eb6aa2c0e2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:03.654205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:03.718941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1227/2446] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:27:03.724105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7840a4-e0c5-4bac-a125-eb6aa2c0e2a7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.747527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1220/2447] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:03 2022] PUT /identity/v3/projects/58c44a80e3c449749623917923ecd611/users/b51405380a1e4f369f86ee6d187caff9/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:03.758833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14725ff9-8345-40fc-b275-7470e8dac13d None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.759354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14725ff9-8345-40fc-b275-7470e8dac13d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.760040 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14725ff9-8345-40fc-b275-7470e8dac13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.809802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-14725ff9-8345-40fc-b275-7470e8dac13d None None] MFA Rules not processed for user `b51405380a1e4f369f86ee6d187caff9`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:03.812399 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-14725ff9-8345-40fc-b275-7470e8dac13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:03.831656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7840a4-e0c5-4bac-a125-eb6aa2c0e2a7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:03.893072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776457e7-fc45-4303-b137-9beb3521c7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:03.893686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776457e7-fc45-4303-b137-9beb3521c7ef admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:03.893985 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776457e7-fc45-4303-b137-9beb3521c7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:03.894216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1221/2448] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:03.894688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776457e7-fc45-4303-b137-9beb3521c7ef admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:03.894867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776457e7-fc45-4303-b137-9beb3521c7ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:03.899008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776457e7-fc45-4303-b137-9beb3521c7ef admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:03.908752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14725ff9-8345-40fc-b275-7470e8dac13d None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:03.910823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-14725ff9-8345-40fc-b275-7470e8dac13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:04.000237 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1228/2449] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:03 2022] POST /identity/v3/projects => generated 381 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:04.000893 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14725ff9-8345-40fc-b275-7470e8dac13d None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:04.011370 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776457e7-fc45-4303-b137-9beb3521c7ef admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:04.056818 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37a5c4-9478-4530-8486-5d2d40ae6ff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:04.057275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37a5c4-9478-4530-8486-5d2d40ae6ff1 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.057817 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37a5c4-9478-4530-8486-5d2d40ae6ff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.058052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37a5c4-9478-4530-8486-5d2d40ae6ff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.080357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776457e7-fc45-4303-b137-9beb3521c7ef admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:04.085471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1bc03-1809-47bf-94ce-29a8f0dcb1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:04.086106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1bc03-1809-47bf-94ce-29a8f0dcb1ba admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.086390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1bc03-1809-47bf-94ce-29a8f0dcb1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.086654 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1bc03-1809-47bf-94ce-29a8f0dcb1ba admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.087279 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1bc03-1809-47bf-94ce-29a8f0dcb1ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:04.089206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1bc03-1809-47bf-94ce-29a8f0dcb1ba admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:04.107346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:04.127748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37a5c4-9478-4530-8486-5d2d40ae6ff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:04.129196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37a5c4-9478-4530-8486-5d2d40ae6ff1 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:04.140332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1222/2450] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:27:04.162200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37a5c4-9478-4530-8486-5d2d40ae6ff1 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:04.177332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37a5c4-9478-4530-8486-5d2d40ae6ff1 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:04.182210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:04.182839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.183121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.184108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] PATH_INFO: `/v3/users/b51405380a1e4f369f86ee6d187caff9` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.184533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b51405380a1e4f369f86ee6d187caff9)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:04.187025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:04.224510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:04.283572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1229/2451] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:27:04 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:27:04.287623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1bc03-1809-47bf-94ce-29a8f0dcb1ba admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:04.298634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1bc03-1809-47bf-94ce-29a8f0dcb1ba admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:04.303240 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e9782-594a-49e3-9180-730a02de5491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:04.303829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e9782-594a-49e3-9180-730a02de5491 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.304120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e9782-594a-49e3-9180-730a02de5491 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.304384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e9782-594a-49e3-9180-730a02de5491 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.305009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e9782-594a-49e3-9180-730a02de5491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:04.306629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e9782-594a-49e3-9180-730a02de5491 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:04.318800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1230/2452] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:27:04.323857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e9782-594a-49e3-9180-730a02de5491 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:04.336059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e9782-594a-49e3-9180-730a02de5491 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:04.343626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db2c87-602b-459e-bfff-4e5654521643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:04.349529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db2c87-602b-459e-bfff-4e5654521643 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.349529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db2c87-602b-459e-bfff-4e5654521643 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.349529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db2c87-602b-459e-bfff-4e5654521643 admin admin] PATH_INFO: `/v3/projects/5616191f622c4bed886a13d9413d72f3/users/94d75717a7824de68023e06bcc0c6511/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.363553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db2c87-602b-459e-bfff-4e5654521643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5616191f622c4bed886a13d9413d72f3, user_id=94d75717a7824de68023e06bcc0c6511, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:04.367014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db2c87-602b-459e-bfff-4e5654521643 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:04.415419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b51405380a1e4f369f86ee6d187caff9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:04.496602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b51405380a1e4f369f86ee6d187caff9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:04.503467 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1231/2453] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:04 2022] PUT /identity/v3/projects/5616191f622c4bed886a13d9413d72f3/users/94d75717a7824de68023e06bcc0c6511/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:04.510722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b51405380a1e4f369f86ee6d187caff9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:04.510826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db2c87-602b-459e-bfff-4e5654521643 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:04.529163 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b51405380a1e4f369f86ee6d187caff9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:04.534439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6109eb2f-21b4-4c95-83a9-fd3c2660fa77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b51405380a1e4f369f86ee6d187caff9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:04.546951 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1223/2454] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:27:04 2022] DELETE /identity/v3/users/b51405380a1e4f369f86ee6d187caff9 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:04.548718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e795911-3f13-4149-812f-4abacc8bda29 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.549053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e795911-3f13-4149-812f-4abacc8bda29 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.549330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e795911-3f13-4149-812f-4abacc8bda29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.581675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-7e795911-3f13-4149-812f-4abacc8bda29 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:04.584121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7e795911-3f13-4149-812f-4abacc8bda29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:04.623459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db2c87-602b-459e-bfff-4e5654521643 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:04.649794 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1224/2455] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:04.660577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e795911-3f13-4149-812f-4abacc8bda29 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:04.663555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7e795911-3f13-4149-812f-4abacc8bda29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:04.685385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fa6d1-4539-45d4-a596-11d5fe892d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:04.686662 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fa6d1-4539-45d4-a596-11d5fe892d2f admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.687009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fa6d1-4539-45d4-a596-11d5fe892d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.687350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062fa6d1-4539-45d4-a596-11d5fe892d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.688072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062fa6d1-4539-45d4-a596-11d5fe892d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:04.689727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062fa6d1-4539-45d4-a596-11d5fe892d2f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:04.705399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1232/2456] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:27:04.709355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fa6d1-4539-45d4-a596-11d5fe892d2f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:04.709355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-062fa6d1-4539-45d4-a596-11d5fe892d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:04.721276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062fa6d1-4539-45d4-a596-11d5fe892d2f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:04.727116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c91905-6cf5-4f61-bf4b-048d18b7d6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:04.727781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c91905-6cf5-4f61-bf4b-048d18b7d6b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.728063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c91905-6cf5-4f61-bf4b-048d18b7d6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.737595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c91905-6cf5-4f61-bf4b-048d18b7d6b9 admin admin] PATH_INFO: `/v3/projects/5616191f622c4bed886a13d9413d72f3/users/94d75717a7824de68023e06bcc0c6511/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.737595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c91905-6cf5-4f61-bf4b-048d18b7d6b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5616191f622c4bed886a13d9413d72f3, user_id=94d75717a7824de68023e06bcc0c6511, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:04.737595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c91905-6cf5-4f61-bf4b-048d18b7d6b9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:04.738228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e795911-3f13-4149-812f-4abacc8bda29 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:04.749927 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1233/2457] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:04 2022] PUT /identity/v3/projects/5616191f622c4bed886a13d9413d72f3/users/94d75717a7824de68023e06bcc0c6511/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:04.760626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce78fb6-2ae1-4a79-b50a-a31a5861079c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.760912 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce78fb6-2ae1-4a79-b50a-a31a5861079c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.761189 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce78fb6-2ae1-4a79-b50a-a31a5861079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.794289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9ce78fb6-2ae1-4a79-b50a-a31a5861079c None None] MFA Rules not processed for user `94d75717a7824de68023e06bcc0c6511`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:04.795589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9ce78fb6-2ae1-4a79-b50a-a31a5861079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:04.796532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713b935-d7aa-499d-80fc-f69566e54732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:04.797617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713b935-d7aa-499d-80fc-f69566e54732 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.798111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713b935-d7aa-499d-80fc-f69566e54732 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.798585 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713b935-d7aa-499d-80fc-f69566e54732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.846934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:04.871661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713b935-d7aa-499d-80fc-f69566e54732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:04.874106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713b935-d7aa-499d-80fc-f69566e54732 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:04.885036 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1225/2458] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:27:04.891504 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1234/2459] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:27:04.899499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713b935-d7aa-499d-80fc-f69566e54732 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:04.901246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8713b935-d7aa-499d-80fc-f69566e54732 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:04.964987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713b935-d7aa-499d-80fc-f69566e54732 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:04.969086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc6d8a-b3e4-4b72-8022-149748fe555f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:04.969772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc6d8a-b3e4-4b72-8022-149748fe555f service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:04.970066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc6d8a-b3e4-4b72-8022-149748fe555f service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:04.970336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc6d8a-b3e4-4b72-8022-149748fe555f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:04.981833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc6d8a-b3e4-4b72-8022-149748fe555f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:04.983492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc6d8a-b3e4-4b72-8022-149748fe555f service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:04.995490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1226/2460] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:27:05.279382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:05.502281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce78fb6-2ae1-4a79-b50a-a31a5861079c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:05.503779 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9ce78fb6-2ae1-4a79-b50a-a31a5861079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:05.518844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce78fb6-2ae1-4a79-b50a-a31a5861079c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:05.523646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:05.524370 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:05.530938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:05.530938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] PATH_INFO: `/v3/projects/58c44a80e3c449749623917923ecd611` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:05.530938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c44a80e3c449749623917923ecd611)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:05.530938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:05.703081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c44a80e3c449749623917923ecd611', 'request_id': 'req-ad80050c-f00b-4661-960a-732bf5e6ff15'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:05.833041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c44a80e3c449749623917923ecd611', 'request_id': 'req-ad80050c-f00b-4661-960a-732bf5e6ff15'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:05.840543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c44a80e3c449749623917923ecd611', 'request_id': 'req-ad80050c-f00b-4661-960a-732bf5e6ff15'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:05.851550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] The token cache is being invalidate because project 58c44a80e3c449749623917923ecd611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:05.851836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:05.852485 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-ad80050c-f00b-4661-960a-732bf5e6ff15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:05.853974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1235/2461] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:05 2022] DELETE /identity/v3/projects/58c44a80e3c449749623917923ecd611 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:05.858827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cead26e-f96d-4af4-8041-f25ca6dd3506 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:05.859116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cead26e-f96d-4af4-8041-f25ca6dd3506 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:05.859373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cead26e-f96d-4af4-8041-f25ca6dd3506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:05.880319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-1cead26e-f96d-4af4-8041-f25ca6dd3506 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:05.881525 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1cead26e-f96d-4af4-8041-f25ca6dd3506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:05.940358 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1227/2462] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:05.948326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e569899-38dd-49e5-878b-48a21d7cfe5a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:05.948611 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e569899-38dd-49e5-878b-48a21d7cfe5a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:05.948883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e569899-38dd-49e5-878b-48a21d7cfe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:05.971209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-1e569899-38dd-49e5-878b-48a21d7cfe5a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:05.978201 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-1e569899-38dd-49e5-878b-48a21d7cfe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:05.982163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1236/2463] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:05.987061 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cead26e-f96d-4af4-8041-f25ca6dd3506 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:05.997333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cead26e-f96d-4af4-8041-f25ca6dd3506 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.002393 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac788b-a096-4ed7-8287-55823be5c15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.003016 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac788b-a096-4ed7-8287-55823be5c15e admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.003286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac788b-a096-4ed7-8287-55823be5c15e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.003549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac788b-a096-4ed7-8287-55823be5c15e admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.004295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac788b-a096-4ed7-8287-55823be5c15e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:27:06.004588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac788b-a096-4ed7-8287-55823be5c15e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.006205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac788b-a096-4ed7-8287-55823be5c15e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.020236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1228/2464] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:27:06.024836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e569899-38dd-49e5-878b-48a21d7cfe5a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.035116 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e569899-38dd-49e5-878b-48a21d7cfe5a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.039922 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ad4cb-c172-45fa-90a7-5b1e52d0ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.040601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ad4cb-c172-45fa-90a7-5b1e52d0ed9e admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.040959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ad4cb-c172-45fa-90a7-5b1e52d0ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.041543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ad4cb-c172-45fa-90a7-5b1e52d0ed9e admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.042231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ad4cb-c172-45fa-90a7-5b1e52d0ed9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.044219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ad4cb-c172-45fa-90a7-5b1e52d0ed9e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.221508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1237/2465] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:06 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:27:06.225508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac788b-a096-4ed7-8287-55823be5c15e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.236108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac788b-a096-4ed7-8287-55823be5c15e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.310524 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67f248-f13e-4e6a-88b9-97f89649f521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.311828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67f248-f13e-4e6a-88b9-97f89649f521 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.311935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67f248-f13e-4e6a-88b9-97f89649f521 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.312191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67f248-f13e-4e6a-88b9-97f89649f521 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.312833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67f248-f13e-4e6a-88b9-97f89649f521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.315531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67f248-f13e-4e6a-88b9-97f89649f521 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.320785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ad4cb-c172-45fa-90a7-5b1e52d0ed9e admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.322597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-902ad4cb-c172-45fa-90a7-5b1e52d0ed9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:06.389782 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ad4cb-c172-45fa-90a7-5b1e52d0ed9e admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.394716 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96496e-4e5b-4642-b998-03313bf3e02d tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94d75717a7824de68023e06bcc0c6511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5616191f622c4bed886a13d9413d72f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.400427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96496e-4e5b-4642-b998-03313bf3e02d tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.400427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96496e-4e5b-4642-b998-03313bf3e02d tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.400427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96496e-4e5b-4642-b998-03313bf3e02d tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.400427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96496e-4e5b-4642-b998-03313bf3e02d tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:27:06.400427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96496e-4e5b-4642-b998-03313bf3e02d tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.401607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96496e-4e5b-4642-b998-03313bf3e02d tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.411294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1238/2466] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:27:06.418480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96496e-4e5b-4642-b998-03313bf3e02d tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.430098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96496e-4e5b-4642-b998-03313bf3e02d tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.434978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59aa77-07a0-4808-84a3-ec587c2f195f tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94d75717a7824de68023e06bcc0c6511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5616191f622c4bed886a13d9413d72f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.435585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59aa77-07a0-4808-84a3-ec587c2f195f tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.435882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59aa77-07a0-4808-84a3-ec587c2f195f tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.436168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59aa77-07a0-4808-84a3-ec587c2f195f tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.436909 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59aa77-07a0-4808-84a3-ec587c2f195f tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.438836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59aa77-07a0-4808-84a3-ec587c2f195f tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.458961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1229/2467] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:27:06 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:06.469174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67f248-f13e-4e6a-88b9-97f89649f521 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.479828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67f248-f13e-4e6a-88b9-97f89649f521 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.485248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10400f89-0328-4f8f-9d6d-1b6e9146972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.485915 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10400f89-0328-4f8f-9d6d-1b6e9146972c admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.488925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10400f89-0328-4f8f-9d6d-1b6e9146972c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.488925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10400f89-0328-4f8f-9d6d-1b6e9146972c admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.488925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10400f89-0328-4f8f-9d6d-1b6e9146972c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.488925 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10400f89-0328-4f8f-9d6d-1b6e9146972c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.500670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1230/2468] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:27:06.504548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10400f89-0328-4f8f-9d6d-1b6e9146972c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.517141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10400f89-0328-4f8f-9d6d-1b6e9146972c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.522183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d33b4-afd1-439a-9326-f5cf349474d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.522766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d33b4-afd1-439a-9326-f5cf349474d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.523220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d33b4-afd1-439a-9326-f5cf349474d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.523498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d33b4-afd1-439a-9326-f5cf349474d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.524253 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d33b4-afd1-439a-9326-f5cf349474d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.525842 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d33b4-afd1-439a-9326-f5cf349474d9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.537685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1231/2469] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:27:06.541973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d33b4-afd1-439a-9326-f5cf349474d9 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.553566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d33b4-afd1-439a-9326-f5cf349474d9 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.558847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214dfada-6358-4830-bca7-5d3e1ef1833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.570654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dfada-6358-4830-bca7-5d3e1ef1833b admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.570654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dfada-6358-4830-bca7-5d3e1ef1833b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.570654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214dfada-6358-4830-bca7-5d3e1ef1833b admin admin] PATH_INFO: `/v3/projects/17c263e876c2467cafd02dffbb8c6e75/users/bf6ac3461af74b4694f6a33449b8d42c/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.570974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1239/2470] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:06 2022] POST /identity/v3/projects => generated 355 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:27:06.573330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59aa77-07a0-4808-84a3-ec587c2f195f tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.576955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214dfada-6358-4830-bca7-5d3e1ef1833b admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c263e876c2467cafd02dffbb8c6e75, user_id=bf6ac3461af74b4694f6a33449b8d42c, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.579891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214dfada-6358-4830-bca7-5d3e1ef1833b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.639350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59aa77-07a0-4808-84a3-ec587c2f195f tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.644073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c73ef6-ccba-4062-b16b-192891ff60af tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94d75717a7824de68023e06bcc0c6511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5616191f622c4bed886a13d9413d72f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.644663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c73ef6-ccba-4062-b16b-192891ff60af tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.644937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c73ef6-ccba-4062-b16b-192891ff60af tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.645204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c73ef6-ccba-4062-b16b-192891ff60af tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.645852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c73ef6-ccba-4062-b16b-192891ff60af tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:27:06.646203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c73ef6-ccba-4062-b16b-192891ff60af tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.647756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c73ef6-ccba-4062-b16b-192891ff60af tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.656548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:06.659505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1232/2471] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:06 2022] PUT /identity/v3/projects/17c263e876c2467cafd02dffbb8c6e75/users/bf6ac3461af74b4694f6a33449b8d42c/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:27:06.661811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1240/2472] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:27:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:27:06.671373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6c6f4-44af-4d05-b559-04c01c8febdc None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.671692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6c6f4-44af-4d05-b559-04c01c8febdc None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.673834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6c6f4-44af-4d05-b559-04c01c8febdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.705908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d9f6c6f4-44af-4d05-b559-04c01c8febdc None None] MFA Rules not processed for user `bf6ac3461af74b4694f6a33449b8d42c`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:06.708060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d9f6c6f4-44af-4d05-b559-04c01c8febdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:06.719061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c73ef6-ccba-4062-b16b-192891ff60af tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.729480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c73ef6-ccba-4062-b16b-192891ff60af tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.778178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:06.790607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94d75717a7824de68023e06bcc0c6511', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5616191f622c4bed886a13d9413d72f3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.791436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.791705 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.792142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] PATH_INFO: `/v3/projects/ed0edd0727af4f56a21a1afcf74774d2` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.793490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ed0edd0727af4f56a21a1afcf74774d2)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.794691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1233/2473] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:06.795947 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.798985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6c6f4-44af-4d05-b559-04c01c8febdc None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.864409 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6c6f4-44af-4d05-b559-04c01c8febdc None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:06.869444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8e79a-8c2b-48b2-bdc5-253efa50d4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:06.875156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8e79a-8c2b-48b2-bdc5-253efa50d4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:06.875156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8e79a-8c2b-48b2-bdc5-253efa50d4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:06.875156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8e79a-8c2b-48b2-bdc5-253efa50d4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:06.875156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8e79a-8c2b-48b2-bdc5-253efa50d4f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:06.875156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8e79a-8c2b-48b2-bdc5-253efa50d4f5 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:06.916877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed0edd0727af4f56a21a1afcf74774d2', 'request_id': 'req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:06.968478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1234/2474] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:06 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:27:06.972725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8e79a-8c2b-48b2-bdc5-253efa50d4f5 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:06.984106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed0edd0727af4f56a21a1afcf74774d2', 'request_id': 'req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:06.991703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed0edd0727af4f56a21a1afcf74774d2', 'request_id': 'req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:07.004018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] The token cache is being invalidate because project ed0edd0727af4f56a21a1afcf74774d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:07.004600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:07.005459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:07.007351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1241/2475] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:06 2022] DELETE /identity/v3/projects/ed0edd0727af4f56a21a1afcf74774d2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:07.040273 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8e79a-8c2b-48b2-bdc5-253efa50d4f5 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:07.045622 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde00746-0744-44a0-adf9-11d4b9590ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:07.046422 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde00746-0744-44a0-adf9-11d4b9590ccd admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:07.046766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde00746-0744-44a0-adf9-11d4b9590ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:07.047452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde00746-0744-44a0-adf9-11d4b9590ccd admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:07.048042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde00746-0744-44a0-adf9-11d4b9590ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:07.050124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde00746-0744-44a0-adf9-11d4b9590ccd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:07.164780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1235/2476] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:27:06 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:27:07.171690 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:07.173160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:07.207954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d6996-10c0-4bc2-b402-e89b4ff35d4d None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:07.207954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d6996-10c0-4bc2-b402-e89b4ff35d4d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:07.208209 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d6996-10c0-4bc2-b402-e89b4ff35d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:07.236618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5d7d6996-10c0-4bc2-b402-e89b4ff35d4d None None] MFA Rules not processed for user `6b7457ab864e4df6b10a195f8500c14c`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:07.237869 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5d7d6996-10c0-4bc2-b402-e89b4ff35d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:07.240607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385bde9-3ab8-4dc9-8907-6acdfb70fcbb tempest-VolumeQuotaClassesTest-1535647778 tempest-VolumeQuotaClassesTest-1535647778-project-admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:07.245967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68521fea-e770-4a28-9438-cf9134e20206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:07.246575 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68521fea-e770-4a28-9438-cf9134e20206 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:07.246853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68521fea-e770-4a28-9438-cf9134e20206 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:07.247113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68521fea-e770-4a28-9438-cf9134e20206 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:07.247760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68521fea-e770-4a28-9438-cf9134e20206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:07.250699 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68521fea-e770-4a28-9438-cf9134e20206 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:07.260979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1242/2477] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:27:07.266073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68521fea-e770-4a28-9438-cf9134e20206 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:07.276666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68521fea-e770-4a28-9438-cf9134e20206 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:07.281675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bc8e7-8f51-4d97-b216-353589e9feb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:07.282346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bc8e7-8f51-4d97-b216-353589e9feb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:07.282620 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bc8e7-8f51-4d97-b216-353589e9feb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:07.283933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bc8e7-8f51-4d97-b216-353589e9feb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:07.283933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bc8e7-8f51-4d97-b216-353589e9feb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:07.285472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bc8e7-8f51-4d97-b216-353589e9feb1 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:07.296999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1243/2478] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:27:07.300865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bc8e7-8f51-4d97-b216-353589e9feb1 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:07.312568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1236/2479] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:27:07.312754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bc8e7-8f51-4d97-b216-353589e9feb1 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:07.319622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bd19d-aa6e-4add-b173-2a47ee6f2fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:07.319832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bd19d-aa6e-4add-b173-2a47ee6f2fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:07.320342 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bd19d-aa6e-4add-b173-2a47ee6f2fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:07.320655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0bd19d-aa6e-4add-b173-2a47ee6f2fd8 admin admin] PATH_INFO: `/v3/projects/4dbfc2d4698f41a8bf93739868d4e07c/users/2e0610048f154b6d9457d784b039ccf5/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:07.345539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bd19d-aa6e-4add-b173-2a47ee6f2fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dbfc2d4698f41a8bf93739868d4e07c, user_id=2e0610048f154b6d9457d784b039ccf5, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:07.349014 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0bd19d-aa6e-4add-b173-2a47ee6f2fd8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:07.441867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1244/2480] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:07 2022] PUT /identity/v3/projects/4dbfc2d4698f41a8bf93739868d4e07c/users/2e0610048f154b6d9457d784b039ccf5/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:27:07.454002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f878995-0445-4641-845d-237dc2d9f9f1 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:07.454299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f878995-0445-4641-845d-237dc2d9f9f1 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:07.454563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f878995-0445-4641-845d-237dc2d9f9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:07.494221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8f878995-0445-4641-845d-237dc2d9f9f1 None None] MFA Rules not processed for user `2e0610048f154b6d9457d784b039ccf5`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:07.495782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8f878995-0445-4641-845d-237dc2d9f9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:07.534737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:07.574866 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1237/2481] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:07.580094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bd19d-aa6e-4add-b173-2a47ee6f2fd8 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:07.668021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0bd19d-aa6e-4add-b173-2a47ee6f2fd8 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:07.674245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87338fbd-10c8-4d98-9c96-10fa8037e805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:07.674857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87338fbd-10c8-4d98-9c96-10fa8037e805 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:07.675136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87338fbd-10c8-4d98-9c96-10fa8037e805 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:07.675406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87338fbd-10c8-4d98-9c96-10fa8037e805 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:07.676477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87338fbd-10c8-4d98-9c96-10fa8037e805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:07.678894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87338fbd-10c8-4d98-9c96-10fa8037e805 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:07.847169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1245/2482] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:07 2022] POST /identity/v3/projects => generated 377 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:07.848668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f878995-0445-4641-845d-237dc2d9f9f1 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:07.863766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f878995-0445-4641-845d-237dc2d9f9f1 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:07.864838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87338fbd-10c8-4d98-9c96-10fa8037e805 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:07.866600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-87338fbd-10c8-4d98-9c96-10fa8037e805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:07.923955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74412f4-6f44-4d19-af89-17b47dc71fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:07.924623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74412f4-6f44-4d19-af89-17b47dc71fcd admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:07.924821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74412f4-6f44-4d19-af89-17b47dc71fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:07.925085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74412f4-6f44-4d19-af89-17b47dc71fcd admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:07.925719 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74412f4-6f44-4d19-af89-17b47dc71fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:07.927747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74412f4-6f44-4d19-af89-17b47dc71fcd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:07.934706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87338fbd-10c8-4d98-9c96-10fa8037e805 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:07.939685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:07.940341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:07.940619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:07.940886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] PATH_INFO: `/v3/users/ff2651d4ed9b497fa8aa037cc4dc4ea3` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:07.942066 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff2651d4ed9b497fa8aa037cc4dc4ea3)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:07.943943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:08.067859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1238/2483] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:27:07 2022] POST /identity/v3/users => generated 295 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:08.071998 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74412f4-6f44-4d19-af89-17b47dc71fcd admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:08.082270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74412f4-6f44-4d19-af89-17b47dc71fcd admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:08.087230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2e4ea-0669-4034-89ea-95202b3acadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:08.087820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2e4ea-0669-4034-89ea-95202b3acadb admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:08.088102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2e4ea-0669-4034-89ea-95202b3acadb admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:08.088371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2e4ea-0669-4034-89ea-95202b3acadb admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:08.089006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2e4ea-0669-4034-89ea-95202b3acadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:08.090661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2e4ea-0669-4034-89ea-95202b3acadb admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:08.101418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1239/2484] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:27:08.105515 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2e4ea-0669-4034-89ea-95202b3acadb admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:08.115406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2e4ea-0669-4034-89ea-95202b3acadb admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:08.129436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b014352-9b69-4430-9268-cfa77d4d07aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:08.129436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b014352-9b69-4430-9268-cfa77d4d07aa admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:08.129436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b014352-9b69-4430-9268-cfa77d4d07aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:08.129436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b014352-9b69-4430-9268-cfa77d4d07aa admin admin] PATH_INFO: `/v3/projects/94a2e4729cf44e3c98097341d05cb0ea/users/fe9a4d2a8354401886f414eaf8afad27/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:08.137038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b014352-9b69-4430-9268-cfa77d4d07aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a2e4729cf44e3c98097341d05cb0ea, user_id=fe9a4d2a8354401886f414eaf8afad27, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:08.140280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b014352-9b69-4430-9268-cfa77d4d07aa admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:08.221726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:08.266986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1240/2485] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:08 2022] PUT /identity/v3/projects/94a2e4729cf44e3c98097341d05cb0ea/users/fe9a4d2a8354401886f414eaf8afad27/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:27:08.269664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b014352-9b69-4430-9268-cfa77d4d07aa admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:08.321915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff2651d4ed9b497fa8aa037cc4dc4ea3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:08.360778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b014352-9b69-4430-9268-cfa77d4d07aa admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:08.367193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bfa27-22c1-47c4-bffd-0c9277cf779a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:08.367437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bfa27-22c1-47c4-bffd-0c9277cf779a admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:08.367972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bfa27-22c1-47c4-bffd-0c9277cf779a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:08.368199 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bfa27-22c1-47c4-bffd-0c9277cf779a admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:08.368708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bfa27-22c1-47c4-bffd-0c9277cf779a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:08.370687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bfa27-22c1-47c4-bffd-0c9277cf779a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:08.385921 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1241/2486] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:27:08.387045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bfa27-22c1-47c4-bffd-0c9277cf779a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:08.405140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bfa27-22c1-47c4-bffd-0c9277cf779a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:08.410473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc81110-aabe-42fc-94f4-cf09e55f2394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:08.411210 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc81110-aabe-42fc-94f4-cf09e55f2394 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:08.411553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc81110-aabe-42fc-94f4-cf09e55f2394 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:08.411890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc81110-aabe-42fc-94f4-cf09e55f2394 admin admin] PATH_INFO: `/v3/projects/94a2e4729cf44e3c98097341d05cb0ea/users/fe9a4d2a8354401886f414eaf8afad27/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:08.414300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc81110-aabe-42fc-94f4-cf09e55f2394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94a2e4729cf44e3c98097341d05cb0ea, user_id=fe9a4d2a8354401886f414eaf8afad27, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:08.417184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc81110-aabe-42fc-94f4-cf09e55f2394 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:08.436290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff2651d4ed9b497fa8aa037cc4dc4ea3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:08.450005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1242/2487] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:08 2022] PUT /identity/v3/projects/94a2e4729cf44e3c98097341d05cb0ea/users/fe9a4d2a8354401886f414eaf8afad27/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:08.458559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610d322-48a1-408f-863a-157057beb610 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:08.458851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610d322-48a1-408f-863a-157057beb610 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:08.459255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610d322-48a1-408f-863a-157057beb610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:08.459812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff2651d4ed9b497fa8aa037cc4dc4ea3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:08.460545 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff2651d4ed9b497fa8aa037cc4dc4ea3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:08.461155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3c2277fb-acd4-47e8-ae80-4757a110dc27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff2651d4ed9b497fa8aa037cc4dc4ea3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:08.492191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1246/2488] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:27:07 2022] DELETE /identity/v3/users/ff2651d4ed9b497fa8aa037cc4dc4ea3 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:08.495117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b45f69-1b2b-4652-806e-db8802f9a3a9 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:08.495568 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b45f69-1b2b-4652-806e-db8802f9a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:08.495990 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b45f69-1b2b-4652-806e-db8802f9a3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:08.498819 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8610d322-48a1-408f-863a-157057beb610 None None] MFA Rules not processed for user `fe9a4d2a8354401886f414eaf8afad27`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:08.502329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8610d322-48a1-408f-863a-157057beb610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:08.521115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-13b45f69-1b2b-4652-806e-db8802f9a3a9 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:08.522414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-13b45f69-1b2b-4652-806e-db8802f9a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:08.580583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1243/2489] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:08.588864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1247/2490] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:27:08.599914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610d322-48a1-408f-863a-157057beb610 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:08.601108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b45f69-1b2b-4652-806e-db8802f9a3a9 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:08.601671 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8610d322-48a1-408f-863a-157057beb610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:08.602943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-13b45f69-1b2b-4652-806e-db8802f9a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:08.662795 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b45f69-1b2b-4652-806e-db8802f9a3a9 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:08.664774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610d322-48a1-408f-863a-157057beb610 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:08.666403 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532d1e4-d7d0-40c9-afbc-f36b04f4334a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:08.667001 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532d1e4-d7d0-40c9-afbc-f36b04f4334a service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:08.667273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532d1e4-d7d0-40c9-afbc-f36b04f4334a service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:08.667537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532d1e4-d7d0-40c9-afbc-f36b04f4334a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:08.668400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffe796-1bf4-4196-8523-91bfe24539a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:08.668989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffe796-1bf4-4196-8523-91bfe24539a4 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:08.669244 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4532d1e4-d7d0-40c9-afbc-f36b04f4334a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:08.669338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffe796-1bf4-4196-8523-91bfe24539a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:08.669543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffe796-1bf4-4196-8523-91bfe24539a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:08.679737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffe796-1bf4-4196-8523-91bfe24539a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:08.681149 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffe796-1bf4-4196-8523-91bfe24539a4 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:08.691167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1244/2491] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:27:08.736842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532d1e4-d7d0-40c9-afbc-f36b04f4334a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:08.738370 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532d1e4-d7d0-40c9-afbc-f36b04f4334a service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:08.749152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1248/2492] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:09.123698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffe796-1bf4-4196-8523-91bfe24539a4 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:09.125139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-36ffe796-1bf4-4196-8523-91bfe24539a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:09.138979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffe796-1bf4-4196-8523-91bfe24539a4 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:09.143496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:09.144087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:09.144362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:09.144630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] PATH_INFO: `/v3/projects/87c6e97feec34b228c2419572723f5a3` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:09.145735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c6e97feec34b228c2419572723f5a3)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:09.147700 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:09.309909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c6e97feec34b228c2419572723f5a3', 'request_id': 'req-2f013252-c938-4e9e-b063-2b97e1cc14eb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:09.367250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:09.400607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c6e97feec34b228c2419572723f5a3', 'request_id': 'req-2f013252-c938-4e9e-b063-2b97e1cc14eb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:09.407513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c6e97feec34b228c2419572723f5a3', 'request_id': 'req-2f013252-c938-4e9e-b063-2b97e1cc14eb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:09.417759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] The token cache is being invalidate because project 87c6e97feec34b228c2419572723f5a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:09.418144 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:09.418783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:09.420453 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1245/2493] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:09 2022] DELETE /identity/v3/projects/87c6e97feec34b228c2419572723f5a3 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:09.424470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532d1e4-d7d0-40c9-afbc-f36b04f4334a service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:09.436076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532d1e4-d7d0-40c9-afbc-f36b04f4334a service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:09.500265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:09.500265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:09.500265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:09.500265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] PATH_INFO: `/v3/users/94d75717a7824de68023e06bcc0c6511` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:09.500603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94d75717a7824de68023e06bcc0c6511)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:09.501263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:09.736721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94d75717a7824de68023e06bcc0c6511'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:09.819356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94d75717a7824de68023e06bcc0c6511'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:09.836337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94d75717a7824de68023e06bcc0c6511'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:09.837112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94d75717a7824de68023e06bcc0c6511'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:09.838347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-273fa2bf-5682-4368-9994-b1054067d752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94d75717a7824de68023e06bcc0c6511'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:09.852788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1249/2494] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:27:09 2022] DELETE /identity/v3/users/94d75717a7824de68023e06bcc0c6511 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:10.499923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:10.501377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:10.515037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f013252-c938-4e9e-b063-2b97e1cc14eb admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:10.574045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:10.574648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:10.574928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:10.575202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] PATH_INFO: `/v3/projects/5616191f622c4bed886a13d9413d72f3` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:10.576354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5616191f622c4bed886a13d9413d72f3)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:10.578314 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:10.688955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5616191f622c4bed886a13d9413d72f3', 'request_id': 'req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:10.781850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5616191f622c4bed886a13d9413d72f3', 'request_id': 'req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:10.788572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5616191f622c4bed886a13d9413d72f3', 'request_id': 'req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:10.800416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] The token cache is being invalidate because project 5616191f622c4bed886a13d9413d72f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:10.800702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:10.801478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-2f59fc3e-5bd2-4385-bca9-3c1fbf0da819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:10.803183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1246/2495] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:10 2022] DELETE /identity/v3/projects/5616191f622c4bed886a13d9413d72f3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:10.808521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ef062-bbf9-46b5-8428-4f79737e5125 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:10.809044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ef062-bbf9-46b5-8428-4f79737e5125 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:10.809044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ef062-bbf9-46b5-8428-4f79737e5125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:10.829487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2b4ef062-bbf9-46b5-8428-4f79737e5125 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:10.830731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2b4ef062-bbf9-46b5-8428-4f79737e5125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:10.893371 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1250/2496] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:10.902080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e5fa4-5722-4222-8f2d-2620f55968ad None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:10.902435 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e5fa4-5722-4222-8f2d-2620f55968ad None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:10.902808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e5fa4-5722-4222-8f2d-2620f55968ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:10.923979 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-967e5fa4-5722-4222-8f2d-2620f55968ad None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:10.925222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-967e5fa4-5722-4222-8f2d-2620f55968ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:10.936820 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1247/2497] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:10.938260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ef062-bbf9-46b5-8428-4f79737e5125 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:10.948675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ef062-bbf9-46b5-8428-4f79737e5125 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:10.953271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf4503-a415-4763-b8cf-b61de9706f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:10.953902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf4503-a415-4763-b8cf-b61de9706f31 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:10.954273 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf4503-a415-4763-b8cf-b61de9706f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:10.954547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf4503-a415-4763-b8cf-b61de9706f31 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:10.955213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf4503-a415-4763-b8cf-b61de9706f31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:27:10.955514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf4503-a415-4763-b8cf-b61de9706f31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:10.957125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf4503-a415-4763-b8cf-b61de9706f31 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:10.969770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1251/2498] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:27:10.974239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e5fa4-5722-4222-8f2d-2620f55968ad None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:10.984187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e5fa4-5722-4222-8f2d-2620f55968ad None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:10.991067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b80d4c-a550-4450-9881-07957e395d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:10.995029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b80d4c-a550-4450-9881-07957e395d2c admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:10.995029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b80d4c-a550-4450-9881-07957e395d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:10.995029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b80d4c-a550-4450-9881-07957e395d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:10.995029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b80d4c-a550-4450-9881-07957e395d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:10.995029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b80d4c-a550-4450-9881-07957e395d2c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:11.092544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1248/2499] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:10 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:11.096539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf4503-a415-4763-b8cf-b61de9706f31 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:11.106603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf4503-a415-4763-b8cf-b61de9706f31 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:11.113302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:11.163883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbd58f-287e-4e49-85d8-9ec9c74ed811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:11.164478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbd58f-287e-4e49-85d8-9ec9c74ed811 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:11.164753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbd58f-287e-4e49-85d8-9ec9c74ed811 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:11.165017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbd58f-287e-4e49-85d8-9ec9c74ed811 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:11.165629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbd58f-287e-4e49-85d8-9ec9c74ed811 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:11.167660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbd58f-287e-4e49-85d8-9ec9c74ed811 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:11.255831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1252/2500] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:27:11 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:11.260214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b80d4c-a550-4450-9881-07957e395d2c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:11.271911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b80d4c-a550-4450-9881-07957e395d2c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:11.276630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0b382-7bd5-4c56-a941-ad0021d5cb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:11.277316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0b382-7bd5-4c56-a941-ad0021d5cb68 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:11.277595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0b382-7bd5-4c56-a941-ad0021d5cb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:11.277863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0b382-7bd5-4c56-a941-ad0021d5cb68 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:11.278575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0b382-7bd5-4c56-a941-ad0021d5cb68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:11.280851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0b382-7bd5-4c56-a941-ad0021d5cb68 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:11.291417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1249/2501] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:27:11.295621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbd58f-287e-4e49-85d8-9ec9c74ed811 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:11.306210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbd58f-287e-4e49-85d8-9ec9c74ed811 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:11.311439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cf9e2-25b0-4dfe-8ca4-7fdc30f94c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:11.312059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cf9e2-25b0-4dfe-8ca4-7fdc30f94c85 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:11.312329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cf9e2-25b0-4dfe-8ca4-7fdc30f94c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:11.312600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cf9e2-25b0-4dfe-8ca4-7fdc30f94c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:11.313238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cf9e2-25b0-4dfe-8ca4-7fdc30f94c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:11.314844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646cf9e2-25b0-4dfe-8ca4-7fdc30f94c85 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:11.326624 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1253/2502] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:27:11.336789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0b382-7bd5-4c56-a941-ad0021d5cb68 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:11.347848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0b382-7bd5-4c56-a941-ad0021d5cb68 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:11.353681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:11.354744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ce11e-2b18-4fc0-961b-f13a1e9f21f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:11.355623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ce11e-2b18-4fc0-961b-f13a1e9f21f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:11.355896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ce11e-2b18-4fc0-961b-f13a1e9f21f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:11.356164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ce11e-2b18-4fc0-961b-f13a1e9f21f1 admin admin] PATH_INFO: `/v3/projects/6c0534a36d1349b3b0fccc469f818348/users/4c63f9122c714531b59e574f581bf802/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:11.371881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ce11e-2b18-4fc0-961b-f13a1e9f21f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0534a36d1349b3b0fccc469f818348, user_id=4c63f9122c714531b59e574f581bf802, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:11.374535 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ce11e-2b18-4fc0-961b-f13a1e9f21f1 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:11.441135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1250/2503] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:11 2022] PUT /identity/v3/projects/6c0534a36d1349b3b0fccc469f818348/users/4c63f9122c714531b59e574f581bf802/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:11.453198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dc07-a42a-4fd7-aa4e-488d98bc9336 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:11.453474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dc07-a42a-4fd7-aa4e-488d98bc9336 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:11.453745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dc07-a42a-4fd7-aa4e-488d98bc9336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:11.484134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-fc74dc07-a42a-4fd7-aa4e-488d98bc9336 None None] MFA Rules not processed for user `4c63f9122c714531b59e574f581bf802`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:11.485358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fc74dc07-a42a-4fd7-aa4e-488d98bc9336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:11.550061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1254/2504] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:11.554648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ce11e-2b18-4fc0-961b-f13a1e9f21f1 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:11.625147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ce11e-2b18-4fc0-961b-f13a1e9f21f1 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:11.629800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deab41-1917-41a5-b694-ca1c9be44195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:11.630435 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deab41-1917-41a5-b694-ca1c9be44195 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:11.630711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deab41-1917-41a5-b694-ca1c9be44195 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:11.630980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deab41-1917-41a5-b694-ca1c9be44195 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:11.631602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01deab41-1917-41a5-b694-ca1c9be44195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:11.633489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01deab41-1917-41a5-b694-ca1c9be44195 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:11.806957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1251/2505] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:11 2022] POST /identity/v3/projects => generated 387 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:11.811616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74dc07-a42a-4fd7-aa4e-488d98bc9336 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:11.826325 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74dc07-a42a-4fd7-aa4e-488d98bc9336 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:11.883883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7da7a-f4be-405b-a54d-b6241d2f51f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:11.884380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7da7a-f4be-405b-a54d-b6241d2f51f4 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:11.884650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7da7a-f4be-405b-a54d-b6241d2f51f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:11.884917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7da7a-f4be-405b-a54d-b6241d2f51f4 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:11.885538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7da7a-f4be-405b-a54d-b6241d2f51f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:11.887480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7da7a-f4be-405b-a54d-b6241d2f51f4 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:11.980751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1255/2506] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:27:11 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:11.985244 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deab41-1917-41a5-b694-ca1c9be44195 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:12.003846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deab41-1917-41a5-b694-ca1c9be44195 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:12.010633 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca2a6-94ab-4d71-909a-541cb2ab7f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:12.011214 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca2a6-94ab-4d71-909a-541cb2ab7f27 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:12.011493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca2a6-94ab-4d71-909a-541cb2ab7f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:12.011762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca2a6-94ab-4d71-909a-541cb2ab7f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:12.012543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ca2a6-94ab-4d71-909a-541cb2ab7f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:12.014181 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ca2a6-94ab-4d71-909a-541cb2ab7f27 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:12.031964 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1252/2507] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:27:12.035314 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7da7a-f4be-405b-a54d-b6241d2f51f4 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:12.044450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca2a6-94ab-4d71-909a-541cb2ab7f27 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:12.046931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-658ca2a6-94ab-4d71-909a-541cb2ab7f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:12.047063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7da7a-f4be-405b-a54d-b6241d2f51f4 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:12.052982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59ec3b-11f6-4605-86bd-7a0cbf838e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:12.053650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59ec3b-11f6-4605-86bd-7a0cbf838e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:12.053929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59ec3b-11f6-4605-86bd-7a0cbf838e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:12.054236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59ec3b-11f6-4605-86bd-7a0cbf838e18 admin admin] PATH_INFO: `/v3/projects/92ca5b62e8a74a708e0df437af8e9653/users/1db8ac5b63194378977ad89a72b02ff0/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:12.073338 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59ec3b-11f6-4605-86bd-7a0cbf838e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ca5b62e8a74a708e0df437af8e9653, user_id=1db8ac5b63194378977ad89a72b02ff0, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:12.076398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59ec3b-11f6-4605-86bd-7a0cbf838e18 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:12.105857 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca2a6-94ab-4d71-909a-541cb2ab7f27 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:12.110930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4527a-99e1-465b-a7df-6e48f601c20d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:12.111551 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4527a-99e1-465b-a7df-6e48f601c20d service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:12.111828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4527a-99e1-465b-a7df-6e48f601c20d service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:12.112094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4527a-99e1-465b-a7df-6e48f601c20d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:12.113813 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5aa4527a-99e1-465b-a7df-6e48f601c20d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:12.143582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1256/2508] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:12 2022] PUT /identity/v3/projects/92ca5b62e8a74a708e0df437af8e9653/users/1db8ac5b63194378977ad89a72b02ff0/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:27:12.149937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59ec3b-11f6-4605-86bd-7a0cbf838e18 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:12.174891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4527a-99e1-465b-a7df-6e48f601c20d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:12.176320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4527a-99e1-465b-a7df-6e48f601c20d service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:12.185271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1253/2509] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:27:12.214881 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59ec3b-11f6-4605-86bd-7a0cbf838e18 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:12.222079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516b091-80f7-4c2c-9d5f-66fe466c3397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:12.222674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516b091-80f7-4c2c-9d5f-66fe466c3397 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:12.222949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516b091-80f7-4c2c-9d5f-66fe466c3397 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:12.223213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516b091-80f7-4c2c-9d5f-66fe466c3397 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:12.223850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516b091-80f7-4c2c-9d5f-66fe466c3397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:12.225418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d516b091-80f7-4c2c-9d5f-66fe466c3397 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:12.239337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1257/2510] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:27:12.243097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4527a-99e1-465b-a7df-6e48f601c20d service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:12.256954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4527a-99e1-465b-a7df-6e48f601c20d service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:12.261647 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a72455-8323-4370-9f80-3f639e01cf7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:12.262372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a72455-8323-4370-9f80-3f639e01cf7c admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:12.262610 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a72455-8323-4370-9f80-3f639e01cf7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:12.262878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a72455-8323-4370-9f80-3f639e01cf7c admin admin] PATH_INFO: `/v3/projects/92ca5b62e8a74a708e0df437af8e9653/users/1db8ac5b63194378977ad89a72b02ff0/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:12.264914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a72455-8323-4370-9f80-3f639e01cf7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=92ca5b62e8a74a708e0df437af8e9653, user_id=1db8ac5b63194378977ad89a72b02ff0, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:12.267551 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a72455-8323-4370-9f80-3f639e01cf7c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:12.278986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1254/2511] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:12 2022] PUT /identity/v3/projects/92ca5b62e8a74a708e0df437af8e9653/users/1db8ac5b63194378977ad89a72b02ff0/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:12.294915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00906a0-76e4-4a04-9916-622e05e131a1 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:12.295869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00906a0-76e4-4a04-9916-622e05e131a1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:12.296185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00906a0-76e4-4a04-9916-622e05e131a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:12.326271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d00906a0-76e4-4a04-9916-622e05e131a1 None None] MFA Rules not processed for user `1db8ac5b63194378977ad89a72b02ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:12.327553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d00906a0-76e4-4a04-9916-622e05e131a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:12.406101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1258/2512] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:12.443380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a72455-8323-4370-9f80-3f639e01cf7c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:12.446027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-34a72455-8323-4370-9f80-3f639e01cf7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:12.480600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:12.524663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a72455-8323-4370-9f80-3f639e01cf7c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:12.529094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d50ed-b606-4ce3-a2cc-c51de277095d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:12.531025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d50ed-b606-4ce3-a2cc-c51de277095d service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:12.531025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d50ed-b606-4ce3-a2cc-c51de277095d service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:12.531218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d50ed-b606-4ce3-a2cc-c51de277095d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:12.540929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d50ed-b606-4ce3-a2cc-c51de277095d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:12.542424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d50ed-b606-4ce3-a2cc-c51de277095d service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:12.554128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1255/2513] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:12.687445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:13.773445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00906a0-76e4-4a04-9916-622e05e131a1 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:13.785989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00906a0-76e4-4a04-9916-622e05e131a1 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:13.790126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390b79b1-19f1-474d-b7a7-7653d56e8ab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:13.790829 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b79b1-19f1-474d-b7a7-7653d56e8ab9 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:13.792397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b79b1-19f1-474d-b7a7-7653d56e8ab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:13.792397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b79b1-19f1-474d-b7a7-7653d56e8ab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:13.793866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-390b79b1-19f1-474d-b7a7-7653d56e8ab9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:13.847339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:13.868592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b79b1-19f1-474d-b7a7-7653d56e8ab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:13.870209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390b79b1-19f1-474d-b7a7-7653d56e8ab9 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:13.880519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1259/2514] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:14.021840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d50ed-b606-4ce3-a2cc-c51de277095d service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:14.089887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d50ed-b606-4ce3-a2cc-c51de277095d service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:14.095539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:14.096172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:14.096445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:14.096711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] PATH_INFO: `/v3/users/4c63f9122c714531b59e574f581bf802` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:14.097852 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c63f9122c714531b59e574f581bf802)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:14.099988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:14.193246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:14.352396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c63f9122c714531b59e574f581bf802'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:14.424703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c63f9122c714531b59e574f581bf802'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:14.432421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c63f9122c714531b59e574f581bf802'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:14.449888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c63f9122c714531b59e574f581bf802'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:14.450772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c63f9122c714531b59e574f581bf802'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:14.458395 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1256/2515] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:27:14 2022] DELETE /identity/v3/users/4c63f9122c714531b59e574f581bf802 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:27:14.462931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99203c5-a353-457a-b305-a23309556ed5 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:14.463212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99203c5-a353-457a-b305-a23309556ed5 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:14.463480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99203c5-a353-457a-b305-a23309556ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:14.485686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-e99203c5-a353-457a-b305-a23309556ed5 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:14.487487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e99203c5-a353-457a-b305-a23309556ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:14.554598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1260/2516] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:14.565432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:14.566897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:14.633617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aad98e-9e5e-407f-848a-b2d0aaac4339 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:14.656737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db765b56-2857-437a-aa23-089f16a881f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:14.658237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db765b56-2857-437a-aa23-089f16a881f8 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:14.658774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db765b56-2857-437a-aa23-089f16a881f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:14.659307 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db765b56-2857-437a-aa23-089f16a881f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:14.672832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db765b56-2857-437a-aa23-089f16a881f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:14.675603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db765b56-2857-437a-aa23-089f16a881f8 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:14.698116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1257/2517] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:15.220087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99203c5-a353-457a-b305-a23309556ed5 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:15.221622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e99203c5-a353-457a-b305-a23309556ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:15.235002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99203c5-a353-457a-b305-a23309556ed5 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:15.239819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:15.240757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:15.241412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:15.241412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] PATH_INFO: `/v3/projects/6c0534a36d1349b3b0fccc469f818348` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:15.242508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0534a36d1349b3b0fccc469f818348)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:15.244633 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:15.366701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0534a36d1349b3b0fccc469f818348', 'request_id': 'req-74f9f54e-c34c-44ce-9528-2fca435d8f80'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:15.438611 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0534a36d1349b3b0fccc469f818348', 'request_id': 'req-74f9f54e-c34c-44ce-9528-2fca435d8f80'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:15.446143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0534a36d1349b3b0fccc469f818348', 'request_id': 'req-74f9f54e-c34c-44ce-9528-2fca435d8f80'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:15.457101 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] The token cache is being invalidate because project 6c0534a36d1349b3b0fccc469f818348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:15.457488 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:15.458276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:15.460158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1261/2518] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:15 2022] DELETE /identity/v3/projects/6c0534a36d1349b3b0fccc469f818348 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:15.464969 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db765b56-2857-437a-aa23-089f16a881f8 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:15.474479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db765b56-2857-437a-aa23-089f16a881f8 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:15.534427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:15.535058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:15.535337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:15.535607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] PATH_INFO: `/v3/users/1db8ac5b63194378977ad89a72b02ff0` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:15.536769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1db8ac5b63194378977ad89a72b02ff0)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:15.538812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:15.733821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1db8ac5b63194378977ad89a72b02ff0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:15.877476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1db8ac5b63194378977ad89a72b02ff0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:15.881038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:15.887889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1db8ac5b63194378977ad89a72b02ff0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:15.900649 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1db8ac5b63194378977ad89a72b02ff0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:15.901414 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-6587b4f8-27f2-43c5-bf9e-adb0cbc83f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1db8ac5b63194378977ad89a72b02ff0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:15.908972 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1258/2519] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:27:15 2022] DELETE /identity/v3/users/1db8ac5b63194378977ad89a72b02ff0 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:15.938138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:16.378857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:16.380369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:16.396550 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9f54e-c34c-44ce-9528-2fca435d8f80 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:16.457075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:16.457653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:16.457931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:16.458227 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] PATH_INFO: `/v3/projects/92ca5b62e8a74a708e0df437af8e9653` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:16.459563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92ca5b62e8a74a708e0df437af8e9653)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:16.461468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:16.611653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92ca5b62e8a74a708e0df437af8e9653', 'request_id': 'req-63d6748b-c455-4e27-bb48-ec6cca3ee879'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:16.718319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92ca5b62e8a74a708e0df437af8e9653', 'request_id': 'req-63d6748b-c455-4e27-bb48-ec6cca3ee879'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:16.725304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92ca5b62e8a74a708e0df437af8e9653', 'request_id': 'req-63d6748b-c455-4e27-bb48-ec6cca3ee879'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:16.735525 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] The token cache is being invalidate because project 92ca5b62e8a74a708e0df437af8e9653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:16.735808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:16.736425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-63d6748b-c455-4e27-bb48-ec6cca3ee879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:16.737970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1262/2520] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:16 2022] DELETE /identity/v3/projects/92ca5b62e8a74a708e0df437af8e9653 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:16.743022 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d67060-9c10-452b-bdc5-e0ba71244927 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:16.743517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d67060-9c10-452b-bdc5-e0ba71244927 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:16.743608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d67060-9c10-452b-bdc5-e0ba71244927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:16.763956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d6d67060-9c10-452b-bdc5-e0ba71244927 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:16.765170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d6d67060-9c10-452b-bdc5-e0ba71244927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:16.823627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1259/2521] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:16.832069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24389ce-1d90-4648-98ca-ab3368207caa None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:16.832351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24389ce-1d90-4648-98ca-ab3368207caa None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:16.832620 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24389ce-1d90-4648-98ca-ab3368207caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:16.851914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b24389ce-1d90-4648-98ca-ab3368207caa None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:16.853109 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b24389ce-1d90-4648-98ca-ab3368207caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:16.860997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1263/2522] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:16.865235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d67060-9c10-452b-bdc5-e0ba71244927 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:16.875479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d67060-9c10-452b-bdc5-e0ba71244927 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:16.880009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f587c7-f0a2-4930-bb78-b14003888315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:16.880590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f587c7-f0a2-4930-bb78-b14003888315 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:16.880862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f587c7-f0a2-4930-bb78-b14003888315 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:16.881119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f587c7-f0a2-4930-bb78-b14003888315 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:16.881755 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f587c7-f0a2-4930-bb78-b14003888315 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:27:16.882089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f587c7-f0a2-4930-bb78-b14003888315 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:16.883664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f587c7-f0a2-4930-bb78-b14003888315 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:16.895837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1260/2523] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:27:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:27:16.900125 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24389ce-1d90-4648-98ca-ab3368207caa None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:16.909272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24389ce-1d90-4648-98ca-ab3368207caa None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:16.913914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb7ada-1879-4402-b68a-c30498074f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:16.914513 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb7ada-1879-4402-b68a-c30498074f28 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:16.914788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb7ada-1879-4402-b68a-c30498074f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:16.915056 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb7ada-1879-4402-b68a-c30498074f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:16.915984 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb7ada-1879-4402-b68a-c30498074f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:16.917540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb7ada-1879-4402-b68a-c30498074f28 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:16.963919 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:17.040395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1264/2524] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:16 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:17.042156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f587c7-f0a2-4930-bb78-b14003888315 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:17.052680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f587c7-f0a2-4930-bb78-b14003888315 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:17.111070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f12d06-b431-457e-9ce0-69958da2fa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:17.111658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12d06-b431-457e-9ce0-69958da2fa89 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:17.111933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12d06-b431-457e-9ce0-69958da2fa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:17.112199 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f12d06-b431-457e-9ce0-69958da2fa89 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:17.112823 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f12d06-b431-457e-9ce0-69958da2fa89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:17.114739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f12d06-b431-457e-9ce0-69958da2fa89 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:17.212662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1261/2525] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:27:17 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:17.217350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb7ada-1879-4402-b68a-c30498074f28 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:17.227712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb7ada-1879-4402-b68a-c30498074f28 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:17.232350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459caa9-f9ef-45ba-b589-d03f32d6fe08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:17.232934 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459caa9-f9ef-45ba-b589-d03f32d6fe08 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:17.233210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459caa9-f9ef-45ba-b589-d03f32d6fe08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:17.233475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459caa9-f9ef-45ba-b589-d03f32d6fe08 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:17.234169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459caa9-f9ef-45ba-b589-d03f32d6fe08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:17.235736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459caa9-f9ef-45ba-b589-d03f32d6fe08 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:17.245410 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1265/2526] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:27:17.249639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f12d06-b431-457e-9ce0-69958da2fa89 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:17.259682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f12d06-b431-457e-9ce0-69958da2fa89 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:17.264997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccc9c5-459d-4425-9548-6ff7aadc72e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:17.265576 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccc9c5-459d-4425-9548-6ff7aadc72e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:17.265854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccc9c5-459d-4425-9548-6ff7aadc72e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:17.266158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccc9c5-459d-4425-9548-6ff7aadc72e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:17.266795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccc9c5-459d-4425-9548-6ff7aadc72e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:17.268378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccc9c5-459d-4425-9548-6ff7aadc72e0 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:17.279692 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1262/2527] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:27:17.284120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459caa9-f9ef-45ba-b589-d03f32d6fe08 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:17.294819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459caa9-f9ef-45ba-b589-d03f32d6fe08 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:17.299541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47202b5f-7bcd-483c-92bd-9b9a2f8c9204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:17.300212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47202b5f-7bcd-483c-92bd-9b9a2f8c9204 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:17.300489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47202b5f-7bcd-483c-92bd-9b9a2f8c9204 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:17.300758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47202b5f-7bcd-483c-92bd-9b9a2f8c9204 admin admin] PATH_INFO: `/v3/projects/2a544388c65d4b148fb045afb94d5c7d/users/ce97d263be31438bb3ae79f6f918ab93/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:17.316854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47202b5f-7bcd-483c-92bd-9b9a2f8c9204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a544388c65d4b148fb045afb94d5c7d, user_id=ce97d263be31438bb3ae79f6f918ab93, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:17.324457 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47202b5f-7bcd-483c-92bd-9b9a2f8c9204 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:17.372163 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:17.423988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1266/2528] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:17 2022] PUT /identity/v3/projects/2a544388c65d4b148fb045afb94d5c7d/users/ce97d263be31438bb3ae79f6f918ab93/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:17.436255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fb9cd-2450-4a8b-b458-5fd0062bfb70 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:17.436903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fb9cd-2450-4a8b-b458-5fd0062bfb70 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:17.437420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fb9cd-2450-4a8b-b458-5fd0062bfb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:17.472328 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-cf6fb9cd-2450-4a8b-b458-5fd0062bfb70 None None] MFA Rules not processed for user `ce97d263be31438bb3ae79f6f918ab93`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:17.474075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cf6fb9cd-2450-4a8b-b458-5fd0062bfb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:17.550096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1263/2529] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:17.558453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47202b5f-7bcd-483c-92bd-9b9a2f8c9204 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:17.561421 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-47202b5f-7bcd-483c-92bd-9b9a2f8c9204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:17.738146 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:17.750456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47202b5f-7bcd-483c-92bd-9b9a2f8c9204 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:17.754290 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a52675-175a-40d4-83f6-6ed8273790e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:17.754892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a52675-175a-40d4-83f6-6ed8273790e7 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:17.755423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a52675-175a-40d4-83f6-6ed8273790e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:17.755423 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a52675-175a-40d4-83f6-6ed8273790e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:17.766311 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a52675-175a-40d4-83f6-6ed8273790e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:17.768364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a52675-175a-40d4-83f6-6ed8273790e7 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:17.781600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1267/2530] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:18.096218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:18.753244 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:21.253101 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fb9cd-2450-4a8b-b458-5fd0062bfb70 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:21.254657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cf6fb9cd-2450-4a8b-b458-5fd0062bfb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:21.362483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fb9cd-2450-4a8b-b458-5fd0062bfb70 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:21.367161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:21.369990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:21.369990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:21.369990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] PATH_INFO: `/v3/users/bf6ac3461af74b4694f6a33449b8d42c` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:21.369990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf6ac3461af74b4694f6a33449b8d42c)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:21.371534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:21.620224 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf6ac3461af74b4694f6a33449b8d42c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:21.706753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf6ac3461af74b4694f6a33449b8d42c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:21.715310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf6ac3461af74b4694f6a33449b8d42c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:21.728651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf6ac3461af74b4694f6a33449b8d42c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:21.729371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf6ac3461af74b4694f6a33449b8d42c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:21.735275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1264/2531] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:27:21 2022] DELETE /identity/v3/users/bf6ac3461af74b4694f6a33449b8d42c => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:27:21.751738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286f09-edfe-4286-879c-c1f7a0c49d4a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:21.752615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286f09-edfe-4286-879c-c1f7a0c49d4a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:21.752615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34286f09-edfe-4286-879c-c1f7a0c49d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:21.771007 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-34286f09-edfe-4286-879c-c1f7a0c49d4a None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:21.772248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-34286f09-edfe-4286-879c-c1f7a0c49d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:21.776507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:21.831110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1268/2532] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:21.841108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:21.843193 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:21.872043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:21.903310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf65eda-4201-4947-ae05-bb3e84846bf6 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:21.907977 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2e4cd-3d93-4053-b1e4-de932872bef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:21.911889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2e4cd-3d93-4053-b1e4-de932872bef1 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:21.911889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2e4cd-3d93-4053-b1e4-de932872bef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:21.911889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2e4cd-3d93-4053-b1e4-de932872bef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:21.918785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2e4cd-3d93-4053-b1e4-de932872bef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:21.920248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2e4cd-3d93-4053-b1e4-de932872bef1 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:21.930809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1265/2533] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:22.390713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286f09-edfe-4286-879c-c1f7a0c49d4a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:22.396284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-34286f09-edfe-4286-879c-c1f7a0c49d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:22.409155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34286f09-edfe-4286-879c-c1f7a0c49d4a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:22.415374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:22.416292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:22.416727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:22.417144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] PATH_INFO: `/v3/projects/17c263e876c2467cafd02dffbb8c6e75` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:22.418498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c263e876c2467cafd02dffbb8c6e75)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:22.421429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:22.554768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c263e876c2467cafd02dffbb8c6e75', 'request_id': 'req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:22.611012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c263e876c2467cafd02dffbb8c6e75', 'request_id': 'req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:22.617988 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c263e876c2467cafd02dffbb8c6e75', 'request_id': 'req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:22.629484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] The token cache is being invalidate because project 17c263e876c2467cafd02dffbb8c6e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:22.629768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:22.630482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:22.632141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1269/2534] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:22 2022] DELETE /identity/v3/projects/17c263e876c2467cafd02dffbb8c6e75 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:22.635860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2e4cd-3d93-4053-b1e4-de932872bef1 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:22.648565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2e4cd-3d93-4053-b1e4-de932872bef1 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:22.717916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:22.718675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:22.718956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:22.719226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] PATH_INFO: `/v3/users/2e0610048f154b6d9457d784b039ccf5` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:22.720357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e0610048f154b6d9457d784b039ccf5)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:22.722340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:22.936396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e0610048f154b6d9457d784b039ccf5'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:23.020749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e0610048f154b6d9457d784b039ccf5'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:23.028908 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e0610048f154b6d9457d784b039ccf5'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:23.041013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e0610048f154b6d9457d784b039ccf5'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:23.041687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e0610048f154b6d9457d784b039ccf5'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:23.048197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1266/2535] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:27:22 2022] DELETE /identity/v3/users/2e0610048f154b6d9457d784b039ccf5 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:23.636245 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:23.636756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:23.653814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2855b1a-6b2d-4a08-8715-2cacdfebcc08 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:23.751191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:23.751870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:23.752148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:23.752419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] PATH_INFO: `/v3/projects/4dbfc2d4698f41a8bf93739868d4e07c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:23.753552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dbfc2d4698f41a8bf93739868d4e07c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:23.755560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:23.897804 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:23.907773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dbfc2d4698f41a8bf93739868d4e07c', 'request_id': 'req-44e71430-f5d5-4c03-8e5f-516090fa42d9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:23.988614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dbfc2d4698f41a8bf93739868d4e07c', 'request_id': 'req-44e71430-f5d5-4c03-8e5f-516090fa42d9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:23.995857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dbfc2d4698f41a8bf93739868d4e07c', 'request_id': 'req-44e71430-f5d5-4c03-8e5f-516090fa42d9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:24.007214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] The token cache is being invalidate because project 4dbfc2d4698f41a8bf93739868d4e07c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:24.007498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:24.008141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:24.009677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1270/2536] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:23 2022] DELETE /identity/v3/projects/4dbfc2d4698f41a8bf93739868d4e07c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:24.013615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:24.015071 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:24.028976 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f46813-abce-4a8e-ae3e-28d45ae201cb admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:24.072509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:24.087511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:24.088222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:24.088492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:24.088759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] PATH_INFO: `/v3/users/fe9a4d2a8354401886f414eaf8afad27` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:24.089898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe9a4d2a8354401886f414eaf8afad27)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:24.091876 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:24.301473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe9a4d2a8354401886f414eaf8afad27'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:24.387841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe9a4d2a8354401886f414eaf8afad27'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:24.396615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe9a4d2a8354401886f414eaf8afad27'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:24.411496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe9a4d2a8354401886f414eaf8afad27'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:24.412223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-96469522-1eb7-42e6-a99d-13d7d6094b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe9a4d2a8354401886f414eaf8afad27'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:24.418349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1267/2537] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:27:24 2022] DELETE /identity/v3/users/fe9a4d2a8354401886f414eaf8afad27 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:24.899385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:24.900917 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:24.915863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e71430-f5d5-4c03-8e5f-516090fa42d9 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:24.973922 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:24.974566 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:24.974846 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:24.975112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] PATH_INFO: `/v3/projects/94a2e4729cf44e3c98097341d05cb0ea` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:24.976201 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=94a2e4729cf44e3c98097341d05cb0ea)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:24.978229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:25.090103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94a2e4729cf44e3c98097341d05cb0ea', 'request_id': 'req-5fbde408-123f-41fe-bdbf-76820b148d7e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:25.186683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94a2e4729cf44e3c98097341d05cb0ea', 'request_id': 'req-5fbde408-123f-41fe-bdbf-76820b148d7e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:25.194501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94a2e4729cf44e3c98097341d05cb0ea', 'request_id': 'req-5fbde408-123f-41fe-bdbf-76820b148d7e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:25.205477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] The token cache is being invalidate because project 94a2e4729cf44e3c98097341d05cb0ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:27:25.205758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:25.206413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5fbde408-123f-41fe-bdbf-76820b148d7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:27:25.208004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1271/2538] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:27:24 2022] DELETE /identity/v3/projects/94a2e4729cf44e3c98097341d05cb0ea => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:25.212819 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5001cf4-e602-4bb5-acd1-46bfcc927029 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:25.213337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5001cf4-e602-4bb5-acd1-46bfcc927029 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:25.213441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5001cf4-e602-4bb5-acd1-46bfcc927029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:25.232188 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d5001cf4-e602-4bb5-acd1-46bfcc927029 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:25.233371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d5001cf4-e602-4bb5-acd1-46bfcc927029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:25.305624 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1268/2539] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:25.315065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6bf62-6bb3-4a5f-ac44-3ffabeed4d74 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:25.315374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6bf62-6bb3-4a5f-ac44-3ffabeed4d74 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:25.315675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6bf62-6bb3-4a5f-ac44-3ffabeed4d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:25.335456 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-41a6bf62-6bb3-4a5f-ac44-3ffabeed4d74 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:25.337451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-41a6bf62-6bb3-4a5f-ac44-3ffabeed4d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:25.344747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1272/2540] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:25.348929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5001cf4-e602-4bb5-acd1-46bfcc927029 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:25.359352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5001cf4-e602-4bb5-acd1-46bfcc927029 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:25.363885 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad301020-a34b-4fff-a48f-767aff881677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:25.364460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad301020-a34b-4fff-a48f-767aff881677 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:25.364735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad301020-a34b-4fff-a48f-767aff881677 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:25.365000 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad301020-a34b-4fff-a48f-767aff881677 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:25.365630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad301020-a34b-4fff-a48f-767aff881677 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:27:25.365922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad301020-a34b-4fff-a48f-767aff881677 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:25.367527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad301020-a34b-4fff-a48f-767aff881677 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:25.379519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1269/2541] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:27:25.383842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6bf62-6bb3-4a5f-ac44-3ffabeed4d74 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:25.393120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6bf62-6bb3-4a5f-ac44-3ffabeed4d74 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:25.398227 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a88b26-c743-43cd-88fc-f40d5ccda8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:25.398824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a88b26-c743-43cd-88fc-f40d5ccda8a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:25.399092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a88b26-c743-43cd-88fc-f40d5ccda8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:25.399355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a88b26-c743-43cd-88fc-f40d5ccda8a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:25.400033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a88b26-c743-43cd-88fc-f40d5ccda8a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:25.401891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a88b26-c743-43cd-88fc-f40d5ccda8a4 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:25.516157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1273/2542] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:27:25 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:27:25.522667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad301020-a34b-4fff-a48f-767aff881677 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:25.532988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad301020-a34b-4fff-a48f-767aff881677 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:25.585591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:25.597600 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6af23-a8a6-4d1e-801d-eb131af18886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:25.597600 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6af23-a8a6-4d1e-801d-eb131af18886 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:25.597600 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6af23-a8a6-4d1e-801d-eb131af18886 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:25.597600 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6af23-a8a6-4d1e-801d-eb131af18886 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:25.597984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6af23-a8a6-4d1e-801d-eb131af18886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:25.600383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c6af23-a8a6-4d1e-801d-eb131af18886 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:25.683738 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1270/2543] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:27:25 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:27:25.688460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a88b26-c743-43cd-88fc-f40d5ccda8a4 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:25.699263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a88b26-c743-43cd-88fc-f40d5ccda8a4 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:25.704121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491efa7-8750-49ee-9533-2483d5e92704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:25.704633 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491efa7-8750-49ee-9533-2483d5e92704 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:25.704913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491efa7-8750-49ee-9533-2483d5e92704 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:25.705186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c491efa7-8750-49ee-9533-2483d5e92704 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:25.705822 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491efa7-8750-49ee-9533-2483d5e92704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:25.707412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c491efa7-8750-49ee-9533-2483d5e92704 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:25.717792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1274/2544] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:27:25.721724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6af23-a8a6-4d1e-801d-eb131af18886 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:25.731935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c6af23-a8a6-4d1e-801d-eb131af18886 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:25.737359 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998eb8c8-8603-4df8-9156-25522481c5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:25.737945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eb8c8-8603-4df8-9156-25522481c5ab admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:25.738254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eb8c8-8603-4df8-9156-25522481c5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:25.738520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998eb8c8-8603-4df8-9156-25522481c5ab admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:25.739161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998eb8c8-8603-4df8-9156-25522481c5ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:25.740731 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998eb8c8-8603-4df8-9156-25522481c5ab admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:25.752370 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1271/2545] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:27:25.756401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491efa7-8750-49ee-9533-2483d5e92704 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:25.763272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:25.767171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c491efa7-8750-49ee-9533-2483d5e92704 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:25.771787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd9c91-9d93-45de-aa47-fad471f7677a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:25.774460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd9c91-9d93-45de-aa47-fad471f7677a admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:25.774460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd9c91-9d93-45de-aa47-fad471f7677a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:25.774460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd9c91-9d93-45de-aa47-fad471f7677a admin admin] PATH_INFO: `/v3/projects/d889272a57c14664a423859179cb2f29/users/a9b612cf039d4bb9b70d78f9a8b9527c/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:25.791030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd9c91-9d93-45de-aa47-fad471f7677a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d889272a57c14664a423859179cb2f29, user_id=a9b612cf039d4bb9b70d78f9a8b9527c, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:25.793629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd9c91-9d93-45de-aa47-fad471f7677a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:25.925935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1275/2546] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:27:25 2022] PUT /identity/v3/projects/d889272a57c14664a423859179cb2f29/users/a9b612cf039d4bb9b70d78f9a8b9527c/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:27:25.936512 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87da34-fbc1-48a5-8171-dd12ee7dd384 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:25.936796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87da34-fbc1-48a5-8171-dd12ee7dd384 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:25.937062 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87da34-fbc1-48a5-8171-dd12ee7dd384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:25.966404 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ad87da34-fbc1-48a5-8171-dd12ee7dd384 None None] MFA Rules not processed for user `a9b612cf039d4bb9b70d78f9a8b9527c`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:25.967605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ad87da34-fbc1-48a5-8171-dd12ee7dd384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:26.030323 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1272/2547] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:26.041131 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd9c91-9d93-45de-aa47-fad471f7677a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:26.042900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-97fd9c91-9d93-45de-aa47-fad471f7677a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:26.099549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd9c91-9d93-45de-aa47-fad471f7677a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:26.103233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b147c-2639-4e03-8eee-8816c1793b5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:26.103814 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b147c-2639-4e03-8eee-8816c1793b5e service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:26.104083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b147c-2639-4e03-8eee-8816c1793b5e service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:26.104344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b147c-2639-4e03-8eee-8816c1793b5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:26.112329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b147c-2639-4e03-8eee-8816c1793b5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:26.113726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b147c-2639-4e03-8eee-8816c1793b5e service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:26.122365 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1276/2548] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:29.084739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3f88f-eb9e-40e1-8e8a-497c8d8ad887 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:29.084739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3f88f-eb9e-40e1-8e8a-497c8d8ad887 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:29.084739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3f88f-eb9e-40e1-8e8a-497c8d8ad887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:29.103169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-95e3f88f-eb9e-40e1-8e8a-497c8d8ad887 None None] MFA Rules not processed for user `a9b612cf039d4bb9b70d78f9a8b9527c`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:27:29.106941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-95e3f88f-eb9e-40e1-8e8a-497c8d8ad887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:29.112058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1273/2549] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:27:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:27:36.118314 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:39.096577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:27:55.242519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b147c-2639-4e03-8eee-8816c1793b5e service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:55.243977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-172b147c-2639-4e03-8eee-8816c1793b5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:55.304022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b147c-2639-4e03-8eee-8816c1793b5e service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:55.309263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c1b7b-14f3-4a44-9057-ef3bedb43372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:55.309862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c1b7b-14f3-4a44-9057-ef3bedb43372 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:55.310166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c1b7b-14f3-4a44-9057-ef3bedb43372 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:55.310433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c1b7b-14f3-4a44-9057-ef3bedb43372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:55.312200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-de3c1b7b-14f3-4a44-9057-ef3bedb43372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:55.390821 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c1b7b-14f3-4a44-9057-ef3bedb43372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:55.392238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c1b7b-14f3-4a44-9057-ef3bedb43372 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:55.403496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1277/2550] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:57.646814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3f88f-eb9e-40e1-8e8a-497c8d8ad887 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:57.648769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-95e3f88f-eb9e-40e1-8e8a-497c8d8ad887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:57.668081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3f88f-eb9e-40e1-8e8a-497c8d8ad887 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:57.675005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375e20f-fb5e-41df-a353-cc6ecc308df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:57.675735 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375e20f-fb5e-41df-a353-cc6ecc308df0 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:57.676082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375e20f-fb5e-41df-a353-cc6ecc308df0 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:57.676416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375e20f-fb5e-41df-a353-cc6ecc308df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:57.678304 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9375e20f-fb5e-41df-a353-cc6ecc308df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:57.691387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375e20f-fb5e-41df-a353-cc6ecc308df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:57.693226 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375e20f-fb5e-41df-a353-cc6ecc308df0 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:57.704895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1274/2551] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:27:59.544715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c1b7b-14f3-4a44-9057-ef3bedb43372 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:27:59.554493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c1b7b-14f3-4a44-9057-ef3bedb43372 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:27:59.558647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59691d1d-cdf5-4d26-9cff-eaf5667e01ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:27:59.559265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59691d1d-cdf5-4d26-9cff-eaf5667e01ea service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:27:59.559541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59691d1d-cdf5-4d26-9cff-eaf5667e01ea service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:27:59.559808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59691d1d-cdf5-4d26-9cff-eaf5667e01ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:27:59.561857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-59691d1d-cdf5-4d26-9cff-eaf5667e01ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:27:59.573407 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59691d1d-cdf5-4d26-9cff-eaf5667e01ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:27:59.574884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59691d1d-cdf5-4d26-9cff-eaf5667e01ea service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:27:59.584616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1278/2552] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:27:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:28:01.296945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375e20f-fb5e-41df-a353-cc6ecc308df0 service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:01.299279 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9375e20f-fb5e-41df-a353-cc6ecc308df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:01.384994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375e20f-fb5e-41df-a353-cc6ecc308df0 service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:01.390824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:01.392878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:01.393352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:01.393800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] PATH_INFO: `/v3/users/ce97d263be31438bb3ae79f6f918ab93` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:01.395491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce97d263be31438bb3ae79f6f918ab93)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:01.397739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:01.688247 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce97d263be31438bb3ae79f6f918ab93'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:01.786141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce97d263be31438bb3ae79f6f918ab93'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:01.807136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce97d263be31438bb3ae79f6f918ab93'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:01.820461 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce97d263be31438bb3ae79f6f918ab93'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:01.821207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce97d263be31438bb3ae79f6f918ab93'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:01.830508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1275/2553] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:28:01 2022] DELETE /identity/v3/users/ce97d263be31438bb3ae79f6f918ab93 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:01.832681 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339feb31-b64f-4852-99be-259b364419f1 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:01.833166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339feb31-b64f-4852-99be-259b364419f1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:01.833261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339feb31-b64f-4852-99be-259b364419f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:01.857712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-339feb31-b64f-4852-99be-259b364419f1 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:01.859202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-339feb31-b64f-4852-99be-259b364419f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:01.893625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:01.895095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:01.930272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1279/2554] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:01.942529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339feb31-b64f-4852-99be-259b364419f1 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:01.944051 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-339feb31-b64f-4852-99be-259b364419f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:01.977009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:01.978952 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:01.983443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d065459-1ff7-4f28-8803-70bc97925375 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:01.991023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3abe5-518b-4eb5-b248-162f2644b86e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:01.992125 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3abe5-518b-4eb5-b248-162f2644b86e service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:01.992593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3abe5-518b-4eb5-b248-162f2644b86e service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:01.993046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3abe5-518b-4eb5-b248-162f2644b86e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:01.995349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-47f3abe5-518b-4eb5-b248-162f2644b86e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:02.006552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339feb31-b64f-4852-99be-259b364419f1 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:02.009896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3abe5-518b-4eb5-b248-162f2644b86e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:02.010606 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd96dac-4eb6-4496-bacb-f14f1fe6189e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:02.011224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd96dac-4eb6-4496-bacb-f14f1fe6189e service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:02.011496 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd96dac-4eb6-4496-bacb-f14f1fe6189e service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:02.011602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3abe5-518b-4eb5-b248-162f2644b86e service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:02.011756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd96dac-4eb6-4496-bacb-f14f1fe6189e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:02.021000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd96dac-4eb6-4496-bacb-f14f1fe6189e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:02.022557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd96dac-4eb6-4496-bacb-f14f1fe6189e service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:02.026048 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1276/2555] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:28:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:28:02.032428 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1280/2556] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:28:02.599265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3abe5-518b-4eb5-b248-162f2644b86e service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:02.601456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-47f3abe5-518b-4eb5-b248-162f2644b86e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:02.616176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3abe5-518b-4eb5-b248-162f2644b86e service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:02.621041 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:02.621688 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:02.622051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:02.622494 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] PATH_INFO: `/v3/projects/2a544388c65d4b148fb045afb94d5c7d` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:02.624033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a544388c65d4b148fb045afb94d5c7d)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:02.626194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:02.757878 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a544388c65d4b148fb045afb94d5c7d', 'request_id': 'req-49dbfd08-e584-4df6-9810-c4b7f143bd53'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:02.842215 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a544388c65d4b148fb045afb94d5c7d', 'request_id': 'req-49dbfd08-e584-4df6-9810-c4b7f143bd53'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:02.849401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a544388c65d4b148fb045afb94d5c7d', 'request_id': 'req-49dbfd08-e584-4df6-9810-c4b7f143bd53'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:02.860661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] The token cache is being invalidate because project 2a544388c65d4b148fb045afb94d5c7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:28:02.860950 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:02.861604 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-49dbfd08-e584-4df6-9810-c4b7f143bd53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:02.863691 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1277/2557] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:28:02 2022] DELETE /identity/v3/projects/2a544388c65d4b148fb045afb94d5c7d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:02.868486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a5630-08f1-48cb-afdb-b8d0063d8ace None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:02.868946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a5630-08f1-48cb-afdb-b8d0063d8ace None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:02.869059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a5630-08f1-48cb-afdb-b8d0063d8ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:02.888820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ae8a5630-08f1-48cb-afdb-b8d0063d8ace None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:02.890043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ae8a5630-08f1-48cb-afdb-b8d0063d8ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:02.950875 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1281/2558] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:02.959116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671e48b-6ae0-4b26-ad07-4f9a3ddad745 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:02.959394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671e48b-6ae0-4b26-ad07-4f9a3ddad745 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:02.959663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671e48b-6ae0-4b26-ad07-4f9a3ddad745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:02.981086 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5671e48b-6ae0-4b26-ad07-4f9a3ddad745 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:02.982440 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5671e48b-6ae0-4b26-ad07-4f9a3ddad745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:02.991607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1278/2559] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:02.995932 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a5630-08f1-48cb-afdb-b8d0063d8ace None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:03.005548 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a5630-08f1-48cb-afdb-b8d0063d8ace None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:03.010144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e1834-faa7-48ad-a632-22530a981b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:03.010728 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e1834-faa7-48ad-a632-22530a981b37 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:03.011006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e1834-faa7-48ad-a632-22530a981b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:03.011277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e1834-faa7-48ad-a632-22530a981b37 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:03.012000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e1834-faa7-48ad-a632-22530a981b37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:28:03.012297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e1834-faa7-48ad-a632-22530a981b37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:03.013863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6e1834-faa7-48ad-a632-22530a981b37 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:03.025742 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1282/2560] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:28:03.030033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671e48b-6ae0-4b26-ad07-4f9a3ddad745 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:03.039752 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671e48b-6ae0-4b26-ad07-4f9a3ddad745 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:03.044469 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27af3c-2b8d-4975-a381-63b5483068d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:03.045060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27af3c-2b8d-4975-a381-63b5483068d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:03.045343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27af3c-2b8d-4975-a381-63b5483068d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:03.045607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a27af3c-2b8d-4975-a381-63b5483068d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:03.046266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27af3c-2b8d-4975-a381-63b5483068d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:03.048146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a27af3c-2b8d-4975-a381-63b5483068d8 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:03.135287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1279/2561] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:28:03 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:28:03.139414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e1834-faa7-48ad-a632-22530a981b37 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:03.149503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6e1834-faa7-48ad-a632-22530a981b37 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:03.205090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ebabd-5cd5-43da-8841-7ab85a2d8d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:03.205682 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ebabd-5cd5-43da-8841-7ab85a2d8d91 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:03.205974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ebabd-5cd5-43da-8841-7ab85a2d8d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:03.206256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ebabd-5cd5-43da-8841-7ab85a2d8d91 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:03.206860 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ebabd-5cd5-43da-8841-7ab85a2d8d91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:03.208747 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ebabd-5cd5-43da-8841-7ab85a2d8d91 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:03.304785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1283/2562] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:28:03 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:28:03.306006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27af3c-2b8d-4975-a381-63b5483068d8 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:03.317104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a27af3c-2b8d-4975-a381-63b5483068d8 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:03.322097 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e62ee-ec2a-4c36-a8f5-8bbc498c191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:03.322741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e62ee-ec2a-4c36-a8f5-8bbc498c191a admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:03.323052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e62ee-ec2a-4c36-a8f5-8bbc498c191a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:03.323354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e62ee-ec2a-4c36-a8f5-8bbc498c191a admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:03.324010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e62ee-ec2a-4c36-a8f5-8bbc498c191a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:03.325685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e62ee-ec2a-4c36-a8f5-8bbc498c191a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:03.335759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1280/2563] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:28:03.339719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ebabd-5cd5-43da-8841-7ab85a2d8d91 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:03.350084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ebabd-5cd5-43da-8841-7ab85a2d8d91 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:03.354688 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a4040-6bf5-489e-ae79-219a65b362cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:03.355353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a4040-6bf5-489e-ae79-219a65b362cb admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:03.355634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a4040-6bf5-489e-ae79-219a65b362cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:03.355904 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a4040-6bf5-489e-ae79-219a65b362cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:03.356538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a4040-6bf5-489e-ae79-219a65b362cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:03.358127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a4040-6bf5-489e-ae79-219a65b362cb admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:03.369185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1284/2564] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:28:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:28:03.373328 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e62ee-ec2a-4c36-a8f5-8bbc498c191a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:03.383322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e62ee-ec2a-4c36-a8f5-8bbc498c191a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:03.388150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139942-23d7-42b9-bc5c-074c100ce116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:03.388722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139942-23d7-42b9-bc5c-074c100ce116 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:03.389002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139942-23d7-42b9-bc5c-074c100ce116 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:03.389274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139942-23d7-42b9-bc5c-074c100ce116 admin admin] PATH_INFO: `/v3/projects/176d15e27cd84f1fbf9bc0cea87535f6/users/761e0ecb20b04c10ac756d3f4b9b0b25/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:03.404761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33139942-23d7-42b9-bc5c-074c100ce116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=176d15e27cd84f1fbf9bc0cea87535f6, user_id=761e0ecb20b04c10ac756d3f4b9b0b25, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:03.407623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33139942-23d7-42b9-bc5c-074c100ce116 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:03.499055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1281/2565] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:28:03 2022] PUT /identity/v3/projects/176d15e27cd84f1fbf9bc0cea87535f6/users/761e0ecb20b04c10ac756d3f4b9b0b25/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:03.510487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b9d5f-605c-47dd-9d70-3585feb01699 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:03.510771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b9d5f-605c-47dd-9d70-3585feb01699 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:03.511048 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b9d5f-605c-47dd-9d70-3585feb01699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:03.540301 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-480b9d5f-605c-47dd-9d70-3585feb01699 None None] MFA Rules not processed for user `761e0ecb20b04c10ac756d3f4b9b0b25`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:03.541512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-480b9d5f-605c-47dd-9d70-3585feb01699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:03.569258 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:03.606345 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1285/2566] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:03.610828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139942-23d7-42b9-bc5c-074c100ce116 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:03.670835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139942-23d7-42b9-bc5c-074c100ce116 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:03.675928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8e225-a924-4852-b6a2-d32e3ede88dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:03.676513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8e225-a924-4852-b6a2-d32e3ede88dd admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:03.676788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8e225-a924-4852-b6a2-d32e3ede88dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:03.677055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8e225-a924-4852-b6a2-d32e3ede88dd admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:03.677665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8e225-a924-4852-b6a2-d32e3ede88dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:03.679562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8e225-a924-4852-b6a2-d32e3ede88dd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:03.694460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:03.839117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1282/2567] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:28:03 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:28:03.843415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b9d5f-605c-47dd-9d70-3585feb01699 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:03.857175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b9d5f-605c-47dd-9d70-3585feb01699 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:03.924702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5e3f2-9e11-4a2d-b6b7-0ab04241b82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:03.925407 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5e3f2-9e11-4a2d-b6b7-0ab04241b82b admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:03.925653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5e3f2-9e11-4a2d-b6b7-0ab04241b82b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:03.925924 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5e3f2-9e11-4a2d-b6b7-0ab04241b82b admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:03.926597 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5e3f2-9e11-4a2d-b6b7-0ab04241b82b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:03.928598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5e3f2-9e11-4a2d-b6b7-0ab04241b82b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:04.037709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1286/2568] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:28:03 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:28:04.049776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8e225-a924-4852-b6a2-d32e3ede88dd admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:04.060148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8e225-a924-4852-b6a2-d32e3ede88dd admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:04.065087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf184414-fc73-4c80-b6b6-ad8b83f61e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:04.065682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf184414-fc73-4c80-b6b6-ad8b83f61e05 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:04.065991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf184414-fc73-4c80-b6b6-ad8b83f61e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:04.066270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf184414-fc73-4c80-b6b6-ad8b83f61e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:04.067280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf184414-fc73-4c80-b6b6-ad8b83f61e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:04.068974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf184414-fc73-4c80-b6b6-ad8b83f61e05 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:04.080668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1283/2569] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:28:04.085618 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5e3f2-9e11-4a2d-b6b7-0ab04241b82b admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:04.097470 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5e3f2-9e11-4a2d-b6b7-0ab04241b82b admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:04.102714 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9efbc90-eb40-4015-b846-2d611de4a6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:04.103319 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efbc90-eb40-4015-b846-2d611de4a6ca admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:04.103595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efbc90-eb40-4015-b846-2d611de4a6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:04.103884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efbc90-eb40-4015-b846-2d611de4a6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:04.104631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9efbc90-eb40-4015-b846-2d611de4a6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:04.106285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9efbc90-eb40-4015-b846-2d611de4a6ca admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:04.118085 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1287/2570] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:28:04.121477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf184414-fc73-4c80-b6b6-ad8b83f61e05 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:04.133694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf184414-fc73-4c80-b6b6-ad8b83f61e05 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:04.138685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ab548-3d3a-4b4d-8578-faa53747a961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:04.139443 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ab548-3d3a-4b4d-8578-faa53747a961 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:04.139737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ab548-3d3a-4b4d-8578-faa53747a961 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:04.140054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947ab548-3d3a-4b4d-8578-faa53747a961 admin admin] PATH_INFO: `/v3/projects/25d486341c164ed89fa03f3b4b5a2c05/users/7be86865a5a1489780981f13e4453c89/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:04.155729 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ab548-3d3a-4b4d-8578-faa53747a961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d486341c164ed89fa03f3b4b5a2c05, user_id=7be86865a5a1489780981f13e4453c89, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:04.158478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947ab548-3d3a-4b4d-8578-faa53747a961 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:04.266294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1284/2571] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:28:04 2022] PUT /identity/v3/projects/25d486341c164ed89fa03f3b4b5a2c05/users/7be86865a5a1489780981f13e4453c89/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:04.277032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816c0a5-eb58-4862-8546-6dcc6d88431a None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:04.277317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816c0a5-eb58-4862-8546-6dcc6d88431a None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:04.277638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816c0a5-eb58-4862-8546-6dcc6d88431a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:04.305679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2816c0a5-eb58-4862-8546-6dcc6d88431a None None] MFA Rules not processed for user `7be86865a5a1489780981f13e4453c89`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:04.306940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2816c0a5-eb58-4862-8546-6dcc6d88431a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:04.378511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1288/2572] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:04.383118 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ab548-3d3a-4b4d-8578-faa53747a961 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:04.451431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947ab548-3d3a-4b4d-8578-faa53747a961 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:04.459473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7261a-99f0-44ba-a95b-44e527eaa512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:04.460082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7261a-99f0-44ba-a95b-44e527eaa512 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:04.460356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7261a-99f0-44ba-a95b-44e527eaa512 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:04.460625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7261a-99f0-44ba-a95b-44e527eaa512 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:04.461287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7261a-99f0-44ba-a95b-44e527eaa512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:04.463317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7261a-99f0-44ba-a95b-44e527eaa512 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:04.601768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1285/2573] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:28:04 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:28:04.603099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816c0a5-eb58-4862-8546-6dcc6d88431a None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:04.614374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816c0a5-eb58-4862-8546-6dcc6d88431a None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:04.658476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:04.687682 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe3a30-e0ad-48fe-b8e8-7f757c932b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:04.688310 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe3a30-e0ad-48fe-b8e8-7f757c932b65 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:04.688583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe3a30-e0ad-48fe-b8e8-7f757c932b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:04.688853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe3a30-e0ad-48fe-b8e8-7f757c932b65 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:04.689477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe3a30-e0ad-48fe-b8e8-7f757c932b65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:04.692286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe3a30-e0ad-48fe-b8e8-7f757c932b65 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:04.830662 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1289/2574] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:28:04 2022] POST /identity/v3/users => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:28:04.834703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7261a-99f0-44ba-a95b-44e527eaa512 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:04.844995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7261a-99f0-44ba-a95b-44e527eaa512 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:04.850073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcce982-d337-44a7-9b3f-22a01eccebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:04.850651 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcce982-d337-44a7-9b3f-22a01eccebbd admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:04.850924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcce982-d337-44a7-9b3f-22a01eccebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:04.851192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcce982-d337-44a7-9b3f-22a01eccebbd admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:04.851829 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcce982-d337-44a7-9b3f-22a01eccebbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:04.853403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcce982-d337-44a7-9b3f-22a01eccebbd admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:04.864916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1286/2575] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:28:04.869128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe3a30-e0ad-48fe-b8e8-7f757c932b65 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:04.879452 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe3a30-e0ad-48fe-b8e8-7f757c932b65 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:04.884092 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331fad8-76d8-4049-a400-613cc2b328de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:04.884749 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331fad8-76d8-4049-a400-613cc2b328de admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:04.885028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331fad8-76d8-4049-a400-613cc2b328de admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:04.885298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331fad8-76d8-4049-a400-613cc2b328de admin admin] PATH_INFO: `/v3/projects/b081ef6394674adea9612b9c767496a7/users/2229f9b3672346f6a6b4aef469c8e9bb/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:04.900017 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331fad8-76d8-4049-a400-613cc2b328de admin admin] RBAC: Authorizing `identity:create_grant(project_id=b081ef6394674adea9612b9c767496a7, user_id=2229f9b3672346f6a6b4aef469c8e9bb, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:04.902628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331fad8-76d8-4049-a400-613cc2b328de admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:05.067446 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1290/2576] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:28:04 2022] PUT /identity/v3/projects/b081ef6394674adea9612b9c767496a7/users/2229f9b3672346f6a6b4aef469c8e9bb/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:28:05.077815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcce982-d337-44a7-9b3f-22a01eccebbd admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:05.092160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcce982-d337-44a7-9b3f-22a01eccebbd admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:05.190550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644cb6f-8c9c-49d4-9d69-5b1d02d39461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:05.190550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644cb6f-8c9c-49d4-9d69-5b1d02d39461 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:05.190550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644cb6f-8c9c-49d4-9d69-5b1d02d39461 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:05.190550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644cb6f-8c9c-49d4-9d69-5b1d02d39461 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:05.191137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644cb6f-8c9c-49d4-9d69-5b1d02d39461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:05.193104 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644cb6f-8c9c-49d4-9d69-5b1d02d39461 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:05.203788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1287/2577] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:28:05.208429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331fad8-76d8-4049-a400-613cc2b328de admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:05.220515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331fad8-76d8-4049-a400-613cc2b328de admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:05.225725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc746d9-c258-4f9a-8b7d-beb721ca9ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:05.226477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc746d9-c258-4f9a-8b7d-beb721ca9ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:05.226740 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc746d9-c258-4f9a-8b7d-beb721ca9ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:05.227005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc746d9-c258-4f9a-8b7d-beb721ca9ea9 admin admin] PATH_INFO: `/v3/projects/b081ef6394674adea9612b9c767496a7/users/2229f9b3672346f6a6b4aef469c8e9bb/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:05.228930 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc746d9-c258-4f9a-8b7d-beb721ca9ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b081ef6394674adea9612b9c767496a7, user_id=2229f9b3672346f6a6b4aef469c8e9bb, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:05.231605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc746d9-c258-4f9a-8b7d-beb721ca9ea9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:05.243454 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1291/2578] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:28:05 2022] PUT /identity/v3/projects/b081ef6394674adea9612b9c767496a7/users/2229f9b3672346f6a6b4aef469c8e9bb/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:05.253849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a702c0-d1be-4b3a-a12b-a5db4b4bfb7b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:05.254167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a702c0-d1be-4b3a-a12b-a5db4b4bfb7b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:05.254441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a702c0-d1be-4b3a-a12b-a5db4b4bfb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:05.283063 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-97a702c0-d1be-4b3a-a12b-a5db4b4bfb7b None None] MFA Rules not processed for user `2229f9b3672346f6a6b4aef469c8e9bb`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:05.285274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-97a702c0-d1be-4b3a-a12b-a5db4b4bfb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:05.305311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:05.360830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1288/2579] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:05.372657 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc746d9-c258-4f9a-8b7d-beb721ca9ea9 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:05.374388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-cdc746d9-c258-4f9a-8b7d-beb721ca9ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:05.434108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc746d9-c258-4f9a-8b7d-beb721ca9ea9 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:05.437777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493b3ee-9731-4ce8-80b6-228cc8290271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:05.438425 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493b3ee-9731-4ce8-80b6-228cc8290271 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:05.438702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493b3ee-9731-4ce8-80b6-228cc8290271 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:05.438971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493b3ee-9731-4ce8-80b6-228cc8290271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:05.499057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493b3ee-9731-4ce8-80b6-228cc8290271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:05.500464 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493b3ee-9731-4ce8-80b6-228cc8290271 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:05.510005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1292/2580] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:28:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:28:07.718665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a702c0-d1be-4b3a-a12b-a5db4b4bfb7b None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:07.732311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a702c0-d1be-4b3a-a12b-a5db4b4bfb7b None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:07.733549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12644f-0500-4c8b-9439-4d30d58ed5d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:07.734173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12644f-0500-4c8b-9439-4d30d58ed5d2 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:07.734450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12644f-0500-4c8b-9439-4d30d58ed5d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:07.734716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a12644f-0500-4c8b-9439-4d30d58ed5d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:07.735885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fc431-f2d8-4d95-9de4-77cd39e8c437 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:07.736344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fc431-f2d8-4d95-9de4-77cd39e8c437 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:07.736427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fc431-f2d8-4d95-9de4-77cd39e8c437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:07.793011 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12644f-0500-4c8b-9439-4d30d58ed5d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:07.794603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a12644f-0500-4c8b-9439-4d30d58ed5d2 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:07.803082 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-e56fc431-f2d8-4d95-9de4-77cd39e8c437 None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:07.804002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1289/2581] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:28:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:28:07.804412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e56fc431-f2d8-4d95-9de4-77cd39e8c437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:07.809185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:07.874175 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1293/2582] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 17:28:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:28:07.983444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12644f-0500-4c8b-9439-4d30d58ed5d2 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:07.984909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2a12644f-0500-4c8b-9439-4d30d58ed5d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:08.079407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a12644f-0500-4c8b-9439-4d30d58ed5d2 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:08.085505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329416e9-d419-4f9d-99d5-a16541643766 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:08.086161 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329416e9-d419-4f9d-99d5-a16541643766 service swift] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:08.086441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329416e9-d419-4f9d-99d5-a16541643766 service swift] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:08.086712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329416e9-d419-4f9d-99d5-a16541643766 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:08.095385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329416e9-d419-4f9d-99d5-a16541643766 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:08.096959 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329416e9-d419-4f9d-99d5-a16541643766 service swift] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:08.105938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1290/2583] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 17:28:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:28:08.981808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19eeef1-961a-43fd-ac33-759ddd89bafb None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:08.982300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19eeef1-961a-43fd-ac33-759ddd89bafb None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:08.982465 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19eeef1-961a-43fd-ac33-759ddd89bafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:09.003050 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-e19eeef1-961a-43fd-ac33-759ddd89bafb None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:09.004264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e19eeef1-961a-43fd-ac33-759ddd89bafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:09.011529 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1294/2584] 10.222.0.79 () {58 vars in 967 bytes} [Tue Aug 9 17:28:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:09.024352 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329416e9-d419-4f9d-99d5-a16541643766 service swift] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:09.035255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329416e9-d419-4f9d-99d5-a16541643766 service swift] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:09.040415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59075199-fe0e-46bb-bf15-8943cf624cb7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:09.041021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075199-fe0e-46bb-bf15-8943cf624cb7 service swift] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:09.041292 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075199-fe0e-46bb-bf15-8943cf624cb7 service swift] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:09.041555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075199-fe0e-46bb-bf15-8943cf624cb7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:09.049333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59075199-fe0e-46bb-bf15-8943cf624cb7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:09.050901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59075199-fe0e-46bb-bf15-8943cf624cb7 service swift] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:09.058570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1291/2585] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 17:28:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:28:09.647241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5562612-3bd8-4f65-af17-29722af0b4df None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:09.647520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5562612-3bd8-4f65-af17-29722af0b4df None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:09.647793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5562612-3bd8-4f65-af17-29722af0b4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:09.666984 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c5562612-3bd8-4f65-af17-29722af0b4df None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:09.668173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c5562612-3bd8-4f65-af17-29722af0b4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:09.675272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1295/2586] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 17:28:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:09.687163 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59075199-fe0e-46bb-bf15-8943cf624cb7 service swift] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:09.698751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59075199-fe0e-46bb-bf15-8943cf624cb7 service swift] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:09.703686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c834ce-ff1a-4106-b349-c2a8b1385f3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:09.704268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c834ce-ff1a-4106-b349-c2a8b1385f3d service swift] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:09.704540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c834ce-ff1a-4106-b349-c2a8b1385f3d service swift] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:09.704801 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c834ce-ff1a-4106-b349-c2a8b1385f3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:09.713793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c834ce-ff1a-4106-b349-c2a8b1385f3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:09.715384 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c834ce-ff1a-4106-b349-c2a8b1385f3d service swift] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:09.718831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:09.723782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1292/2587] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 17:28:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:28:10.269193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6d28-1a79-4e9f-8f0b-00b2a26f8471 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:10.269697 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6d28-1a79-4e9f-8f0b-00b2a26f8471 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:10.269808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a6d28-1a79-4e9f-8f0b-00b2a26f8471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:10.289100 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-248a6d28-1a79-4e9f-8f0b-00b2a26f8471 None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:10.290343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-248a6d28-1a79-4e9f-8f0b-00b2a26f8471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:10.297407 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1296/2588] 10.222.0.79 () {58 vars in 967 bytes} [Tue Aug 9 17:28:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:10.310738 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c834ce-ff1a-4106-b349-c2a8b1385f3d service swift] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:10.321221 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c834ce-ff1a-4106-b349-c2a8b1385f3d service swift] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:10.326419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b86c5-4b98-4620-a370-0a6ebceba38c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:10.327009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b86c5-4b98-4620-a370-0a6ebceba38c service swift] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:10.327287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b86c5-4b98-4620-a370-0a6ebceba38c service swift] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:10.327559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b86c5-4b98-4620-a370-0a6ebceba38c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:10.336467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b86c5-4b98-4620-a370-0a6ebceba38c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:10.338083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b86c5-4b98-4620-a370-0a6ebceba38c service swift] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:10.345940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1293/2589] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 17:28:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:28:11.649929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6d28-1a79-4e9f-8f0b-00b2a26f8471 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:11.651458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-248a6d28-1a79-4e9f-8f0b-00b2a26f8471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:11.664074 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a6d28-1a79-4e9f-8f0b-00b2a26f8471 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:11.667776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44f4a3-6030-414e-b894-da93bf968926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:11.668373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44f4a3-6030-414e-b894-da93bf968926 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:11.668645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44f4a3-6030-414e-b894-da93bf968926 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:11.669016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44f4a3-6030-414e-b894-da93bf968926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:11.679754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44f4a3-6030-414e-b894-da93bf968926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:11.681191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d44f4a3-6030-414e-b894-da93bf968926 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:11.691928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1297/2590] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:28:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:28:14.712602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b86c5-4b98-4620-a370-0a6ebceba38c service swift] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:14.714143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0d6b86c5-4b98-4620-a370-0a6ebceba38c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:14.802321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b86c5-4b98-4620-a370-0a6ebceba38c service swift] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:14.809100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:14.809828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:14.810196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:14.810533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] PATH_INFO: `/v3/users/a9b612cf039d4bb9b70d78f9a8b9527c` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:14.811862 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9b612cf039d4bb9b70d78f9a8b9527c)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:14.813863 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:14.999542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9b612cf039d4bb9b70d78f9a8b9527c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:15.134321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9b612cf039d4bb9b70d78f9a8b9527c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:15.144017 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9b612cf039d4bb9b70d78f9a8b9527c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:15.156544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9b612cf039d4bb9b70d78f9a8b9527c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:15.157318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9b612cf039d4bb9b70d78f9a8b9527c'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:15.164145 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1294/2591] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:28:14 2022] DELETE /identity/v3/users/a9b612cf039d4bb9b70d78f9a8b9527c => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:15.168611 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f1406-269a-4c58-97e6-8526a810ed5c None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:15.173490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f1406-269a-4c58-97e6-8526a810ed5c None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:15.173490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f1406-269a-4c58-97e6-8526a810ed5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:15.187872 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d43f1406-269a-4c58-97e6-8526a810ed5c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:15.189037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d43f1406-269a-4c58-97e6-8526a810ed5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:15.250512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1298/2592] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:15.261276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:15.262733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:15.322837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b9b53-ac74-466c-89ec-6a62ba43be5f admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:15.326827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42254fa-1491-4e77-98e1-ed6ef39a2121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:15.327417 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42254fa-1491-4e77-98e1-ed6ef39a2121 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:15.327695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42254fa-1491-4e77-98e1-ed6ef39a2121 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:15.327963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42254fa-1491-4e77-98e1-ed6ef39a2121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:15.338182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42254fa-1491-4e77-98e1-ed6ef39a2121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:15.339766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42254fa-1491-4e77-98e1-ed6ef39a2121 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:15.350487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1295/2593] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:28:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:28:15.609639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f1406-269a-4c58-97e6-8526a810ed5c None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:15.611199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d43f1406-269a-4c58-97e6-8526a810ed5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:15.628692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f1406-269a-4c58-97e6-8526a810ed5c None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:15.633685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:15.634462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:15.634745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:15.635018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] PATH_INFO: `/v3/projects/d889272a57c14664a423859179cb2f29` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:15.636199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] RBAC: Authorizing `identity:delete_project(project_id=d889272a57c14664a423859179cb2f29)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:15.638217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:15.762749 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd889272a57c14664a423859179cb2f29', 'request_id': 'req-020d38db-6710-4281-9c42-71cdc70fc7af'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:15.830032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd889272a57c14664a423859179cb2f29', 'request_id': 'req-020d38db-6710-4281-9c42-71cdc70fc7af'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:15.837206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd889272a57c14664a423859179cb2f29', 'request_id': 'req-020d38db-6710-4281-9c42-71cdc70fc7af'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:15.843211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:15.850603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] The token cache is being invalidate because project d889272a57c14664a423859179cb2f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:28:15.850897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:15.851603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-020d38db-6710-4281-9c42-71cdc70fc7af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:15.853119 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1299/2594] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:28:15 2022] DELETE /identity/v3/projects/d889272a57c14664a423859179cb2f29 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:17.696124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42254fa-1491-4e77-98e1-ed6ef39a2121 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:17.697684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d42254fa-1491-4e77-98e1-ed6ef39a2121 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:17.710756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42254fa-1491-4e77-98e1-ed6ef39a2121 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:17.714359 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:17.768887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:17.769510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:17.769784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:17.770115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] PATH_INFO: `/v3/users/761e0ecb20b04c10ac756d3f4b9b0b25` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:17.771354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=761e0ecb20b04c10ac756d3f4b9b0b25)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:17.773317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:17.969300 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '761e0ecb20b04c10ac756d3f4b9b0b25'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:18.041687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '761e0ecb20b04c10ac756d3f4b9b0b25'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:18.049425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '761e0ecb20b04c10ac756d3f4b9b0b25'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:18.061631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '761e0ecb20b04c10ac756d3f4b9b0b25'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:18.062333 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '761e0ecb20b04c10ac756d3f4b9b0b25'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:18.069356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1296/2595] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:28:17 2022] DELETE /identity/v3/users/761e0ecb20b04c10ac756d3f4b9b0b25 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:18.074093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888914e9-fef6-4ac0-9fdc-74871b71467f None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:18.074569 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888914e9-fef6-4ac0-9fdc-74871b71467f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:18.075045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888914e9-fef6-4ac0-9fdc-74871b71467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:18.094232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-888914e9-fef6-4ac0-9fdc-74871b71467f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:18.095443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-888914e9-fef6-4ac0-9fdc-74871b71467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:18.155516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1300/2596] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:18.165623 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:18.168262 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:18.226115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ba84f-8eb1-4a75-a8bd-fd02a6dbfb14 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:18.229693 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bcfee-2864-4a8d-b71f-5bcc4fa6ae90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:18.230342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bcfee-2864-4a8d-b71f-5bcc4fa6ae90 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:18.230615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bcfee-2864-4a8d-b71f-5bcc4fa6ae90 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:18.230880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bcfee-2864-4a8d-b71f-5bcc4fa6ae90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:18.241390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bcfee-2864-4a8d-b71f-5bcc4fa6ae90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:18.242914 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bcfee-2864-4a8d-b71f-5bcc4fa6ae90 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:18.252356 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1297/2597] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:28:18.701592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888914e9-fef6-4ac0-9fdc-74871b71467f None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:18.703147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-888914e9-fef6-4ac0-9fdc-74871b71467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:18.733282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888914e9-fef6-4ac0-9fdc-74871b71467f None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:18.733282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:18.733282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:18.733282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:18.733992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] PATH_INFO: `/v3/projects/176d15e27cd84f1fbf9bc0cea87535f6` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:18.733992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=176d15e27cd84f1fbf9bc0cea87535f6)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:18.736859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:18.915487 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '176d15e27cd84f1fbf9bc0cea87535f6', 'request_id': 'req-c21c34ae-ab1e-4661-8417-fd2e73a82495'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:19.019769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '176d15e27cd84f1fbf9bc0cea87535f6', 'request_id': 'req-c21c34ae-ab1e-4661-8417-fd2e73a82495'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:19.027997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '176d15e27cd84f1fbf9bc0cea87535f6', 'request_id': 'req-c21c34ae-ab1e-4661-8417-fd2e73a82495'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:19.039660 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] The token cache is being invalidate because project 176d15e27cd84f1fbf9bc0cea87535f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:28:19.040051 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:19.040712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:19.042231 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1301/2598] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:28:18 2022] DELETE /identity/v3/projects/176d15e27cd84f1fbf9bc0cea87535f6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:19.046446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bcfee-2864-4a8d-b71f-5bcc4fa6ae90 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:19.056511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bcfee-2864-4a8d-b71f-5bcc4fa6ae90 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:19.085892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:19.117939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:19.118585 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:19.118860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:19.120096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] PATH_INFO: `/v3/users/7be86865a5a1489780981f13e4453c89` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:19.121253 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be86865a5a1489780981f13e4453c89)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:19.123203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:19.301680 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be86865a5a1489780981f13e4453c89'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:19.394675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be86865a5a1489780981f13e4453c89'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:19.402625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be86865a5a1489780981f13e4453c89'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:19.416612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be86865a5a1489780981f13e4453c89'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:19.417424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be86865a5a1489780981f13e4453c89'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:19.423830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1298/2599] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:28:19 2022] DELETE /identity/v3/users/7be86865a5a1489780981f13e4453c89 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:19.878668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:19.880154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:19.895289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c34ae-ab1e-4661-8417-fd2e73a82495 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:19.965938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:19.966933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:19.967363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:19.967764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] PATH_INFO: `/v3/projects/25d486341c164ed89fa03f3b4b5a2c05` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:19.969856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d486341c164ed89fa03f3b4b5a2c05)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:19.972758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:20.003609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:20.120257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d486341c164ed89fa03f3b4b5a2c05', 'request_id': 'req-5bce4c9e-d6f3-445b-985e-e968ebfc3913'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:20.197021 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d486341c164ed89fa03f3b4b5a2c05', 'request_id': 'req-5bce4c9e-d6f3-445b-985e-e968ebfc3913'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:20.206236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d486341c164ed89fa03f3b4b5a2c05', 'request_id': 'req-5bce4c9e-d6f3-445b-985e-e968ebfc3913'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:20.218991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] The token cache is being invalidate because project 25d486341c164ed89fa03f3b4b5a2c05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:28:20.219274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:20.219959 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:20.222131 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1302/2600] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:28:19 2022] DELETE /identity/v3/projects/25d486341c164ed89fa03f3b4b5a2c05 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:20.226945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:20.228599 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:20.241838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc016f-2823-4057-a00a-e4d2484ddd50 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:20.300665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:20.301286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:20.301566 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:20.301846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] PATH_INFO: `/v3/users/2229f9b3672346f6a6b4aef469c8e9bb` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:20.303065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2229f9b3672346f6a6b4aef469c8e9bb)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:20.305033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:20.552727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2229f9b3672346f6a6b4aef469c8e9bb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:20.557154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:20.675682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2229f9b3672346f6a6b4aef469c8e9bb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:20.683962 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2229f9b3672346f6a6b4aef469c8e9bb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:20.697281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2229f9b3672346f6a6b4aef469c8e9bb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:20.698051 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1566d502-251b-4ce8-be2f-3850168c51b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2229f9b3672346f6a6b4aef469c8e9bb'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:20.703980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1299/2601] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:28:20 2022] DELETE /identity/v3/users/2229f9b3672346f6a6b4aef469c8e9bb => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:21.123000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:21.126287 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:21.147636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce4c9e-d6f3-445b-985e-e968ebfc3913 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:21.209594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:21.209839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:21.209965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:21.210246 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] PATH_INFO: `/v3/projects/b081ef6394674adea9612b9c767496a7` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:21.211453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b081ef6394674adea9612b9c767496a7)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:21.213332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:21.322808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b081ef6394674adea9612b9c767496a7', 'request_id': 'req-331466e0-0c85-43e1-9bff-f8e7db718f72'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:21.389908 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b081ef6394674adea9612b9c767496a7', 'request_id': 'req-331466e0-0c85-43e1-9bff-f8e7db718f72'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:21.398583 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b081ef6394674adea9612b9c767496a7', 'request_id': 'req-331466e0-0c85-43e1-9bff-f8e7db718f72'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:21.410375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] The token cache is being invalidate because project b081ef6394674adea9612b9c767496a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:28:21.410797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:21.411466 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-331466e0-0c85-43e1-9bff-f8e7db718f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:21.412963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1303/2602] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:28:21 2022] DELETE /identity/v3/projects/b081ef6394674adea9612b9c767496a7 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:21.418130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6a52c-69d2-4a3f-8f6d-6db71227abf7 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:21.418613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6a52c-69d2-4a3f-8f6d-6db71227abf7 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:21.418733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6a52c-69d2-4a3f-8f6d-6db71227abf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:21.438951 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-86e6a52c-69d2-4a3f-8f6d-6db71227abf7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:21.440167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-86e6a52c-69d2-4a3f-8f6d-6db71227abf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:21.499331 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1300/2603] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:21.511996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706697f-f569-4bc6-9672-7893cdffaab1 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:21.512489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706697f-f569-4bc6-9672-7893cdffaab1 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:21.512755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706697f-f569-4bc6-9672-7893cdffaab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:21.533147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c706697f-f569-4bc6-9672-7893cdffaab1 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:21.534556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c706697f-f569-4bc6-9672-7893cdffaab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:21.548972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1304/2604] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:21.553867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6a52c-69d2-4a3f-8f6d-6db71227abf7 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:21.563389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6a52c-69d2-4a3f-8f6d-6db71227abf7 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:21.569598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c78731-3616-46ef-a85f-93b2edfc433e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:21.570211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c78731-3616-46ef-a85f-93b2edfc433e admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:21.570487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c78731-3616-46ef-a85f-93b2edfc433e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:21.570756 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c78731-3616-46ef-a85f-93b2edfc433e admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:21.571488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c78731-3616-46ef-a85f-93b2edfc433e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:28:21.571833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c78731-3616-46ef-a85f-93b2edfc433e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:21.573493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c78731-3616-46ef-a85f-93b2edfc433e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:21.584992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1301/2605] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:28:21.589262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706697f-f569-4bc6-9672-7893cdffaab1 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:21.598971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706697f-f569-4bc6-9672-7893cdffaab1 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:21.603577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ced9b8-9b4b-4446-8c8c-31f2779b57bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:21.604156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ced9b8-9b4b-4446-8c8c-31f2779b57bd admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:21.604424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ced9b8-9b4b-4446-8c8c-31f2779b57bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:21.604682 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ced9b8-9b4b-4446-8c8c-31f2779b57bd admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:21.605293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ced9b8-9b4b-4446-8c8c-31f2779b57bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:21.607188 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ced9b8-9b4b-4446-8c8c-31f2779b57bd admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:21.695977 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1305/2606] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:28:21 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:28:21.700268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c78731-3616-46ef-a85f-93b2edfc433e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:21.711441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c78731-3616-46ef-a85f-93b2edfc433e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:21.770570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d3924-404a-429d-8e71-956f7263e0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:21.771176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d3924-404a-429d-8e71-956f7263e0db admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:21.771460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d3924-404a-429d-8e71-956f7263e0db admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:21.771728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d3924-404a-429d-8e71-956f7263e0db admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:21.772383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d3924-404a-429d-8e71-956f7263e0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:21.774330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d3924-404a-429d-8e71-956f7263e0db admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:21.895164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1302/2607] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:28:21 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:28:21.900755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ced9b8-9b4b-4446-8c8c-31f2779b57bd admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:21.911983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ced9b8-9b4b-4446-8c8c-31f2779b57bd admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:21.916683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1a464-e8e6-430e-b8cd-4322681c8f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:21.919419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1a464-e8e6-430e-b8cd-4322681c8f55 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:21.919419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1a464-e8e6-430e-b8cd-4322681c8f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:21.919419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1a464-e8e6-430e-b8cd-4322681c8f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:21.919419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1a464-e8e6-430e-b8cd-4322681c8f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:21.920009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1a464-e8e6-430e-b8cd-4322681c8f55 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:21.925879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:21.931526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1306/2608] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:28:21.940967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d3924-404a-429d-8e71-956f7263e0db admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:21.953142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d3924-404a-429d-8e71-956f7263e0db admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:21.959900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071ec64-e595-45fe-bed5-6b0fb30cccb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:21.960636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071ec64-e595-45fe-bed5-6b0fb30cccb5 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:21.961203 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071ec64-e595-45fe-bed5-6b0fb30cccb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:21.961548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071ec64-e595-45fe-bed5-6b0fb30cccb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:21.965324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071ec64-e595-45fe-bed5-6b0fb30cccb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:21.965324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071ec64-e595-45fe-bed5-6b0fb30cccb5 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:21.976769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1303/2609] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:28:21.981785 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1a464-e8e6-430e-b8cd-4322681c8f55 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:21.992605 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1a464-e8e6-430e-b8cd-4322681c8f55 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:21.998012 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad7310-582c-4f97-86d7-5549040ca112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:21.998656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad7310-582c-4f97-86d7-5549040ca112 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:21.998928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad7310-582c-4f97-86d7-5549040ca112 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:21.999194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad7310-582c-4f97-86d7-5549040ca112 admin admin] PATH_INFO: `/v3/projects/95580e3f037c419d8eba0dbaace0348e/users/1a3814e4962f4e2fb7956b7a19af75d2/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:22.015153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad7310-582c-4f97-86d7-5549040ca112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95580e3f037c419d8eba0dbaace0348e, user_id=1a3814e4962f4e2fb7956b7a19af75d2, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:22.018291 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad7310-582c-4f97-86d7-5549040ca112 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:22.089768 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1307/2610] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:28:21 2022] PUT /identity/v3/projects/95580e3f037c419d8eba0dbaace0348e/users/1a3814e4962f4e2fb7956b7a19af75d2/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:22.101835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8794d-5372-4efe-b00b-a9d463175e2d None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:22.102369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8794d-5372-4efe-b00b-a9d463175e2d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:22.102898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8794d-5372-4efe-b00b-a9d463175e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:22.162472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-95f8794d-5372-4efe-b00b-a9d463175e2d None None] MFA Rules not processed for user `1a3814e4962f4e2fb7956b7a19af75d2`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:28:22.163984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-95f8794d-5372-4efe-b00b-a9d463175e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:22.194940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:22.248668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1304/2611] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:28:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:28:22.264980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad7310-582c-4f97-86d7-5549040ca112 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:22.266831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-07ad7310-582c-4f97-86d7-5549040ca112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:22.336411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad7310-582c-4f97-86d7-5549040ca112 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:22.341457 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c6ab0-c701-45e0-a84b-2e1430c8638b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33003a9bf4454f5bac2e6ad73544efc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:22.342564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c6ab0-c701-45e0-a84b-2e1430c8638b service glance] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:22.343042 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c6ab0-c701-45e0-a84b-2e1430c8638b service glance] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:22.343317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c6ab0-c701-45e0-a84b-2e1430c8638b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:22.356145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c6ab0-c701-45e0-a84b-2e1430c8638b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:22.357775 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c6ab0-c701-45e0-a84b-2e1430c8638b service glance] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:22.372628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1308/2612] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:28:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:28:32.364549 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:32.486764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=17 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:43.357797 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8794d-5372-4efe-b00b-a9d463175e2d None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:43.359299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-95f8794d-5372-4efe-b00b-a9d463175e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:43.423143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8794d-5372-4efe-b00b-a9d463175e2d None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:43.427515 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eeac13-2c37-43d5-9987-5e8b8fb3d6fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:43.427983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eeac13-2c37-43d5-9987-5e8b8fb3d6fc service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:43.428255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eeac13-2c37-43d5-9987-5e8b8fb3d6fc service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:43.428518 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eeac13-2c37-43d5-9987-5e8b8fb3d6fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:43.430635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d6eeac13-2c37-43d5-9987-5e8b8fb3d6fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:43.518212 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eeac13-2c37-43d5-9987-5e8b8fb3d6fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:43.520563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eeac13-2c37-43d5-9987-5e8b8fb3d6fc service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:43.537770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1305/2613] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:28:44.093855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c6ab0-c701-45e0-a84b-2e1430c8638b service glance] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:44.095989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d51c6ab0-c701-45e0-a84b-2e1430c8638b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:44.176229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c6ab0-c701-45e0-a84b-2e1430c8638b service glance] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:44.181636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b49a54-ea9f-463b-b94c-d89aa4f577b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:44.184629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b49a54-ea9f-463b-b94c-d89aa4f577b9 service placement] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:44.184629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b49a54-ea9f-463b-b94c-d89aa4f577b9 service placement] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:44.184629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b49a54-ea9f-463b-b94c-d89aa4f577b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:44.185009 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d3b49a54-ea9f-463b-b94c-d89aa4f577b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:44.197676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b49a54-ea9f-463b-b94c-d89aa4f577b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:44.199309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b49a54-ea9f-463b-b94c-d89aa4f577b9 service placement] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:44.211122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1309/2614] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:28:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:28:44.252744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eeac13-2c37-43d5-9987-5e8b8fb3d6fc service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:44.256516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d6eeac13-2c37-43d5-9987-5e8b8fb3d6fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:28:44.267782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eeac13-2c37-43d5-9987-5e8b8fb3d6fc service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:44.274037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:44.274694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:44.274961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:44.275229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] PATH_INFO: `/v3/users/f493a8495dd64a94b12e159862167408` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:44.276389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f493a8495dd64a94b12e159862167408)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:44.278502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:44.515213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f493a8495dd64a94b12e159862167408'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:44.605380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f493a8495dd64a94b12e159862167408'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:44.616590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f493a8495dd64a94b12e159862167408'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:44.630787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f493a8495dd64a94b12e159862167408'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:44.631547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f493a8495dd64a94b12e159862167408'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:44.637657 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1306/2615] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:28:44 2022] DELETE /identity/v3/users/f493a8495dd64a94b12e159862167408 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:44.917764 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b49a54-ea9f-463b-b94c-d89aa4f577b9 service placement] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:28:44.929241 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b49a54-ea9f-463b-b94c-d89aa4f577b9 service placement] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:28:45.002126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:28:45.002864 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:28:45.003194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:28:45.003578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] PATH_INFO: `/v3/projects/883aca80f1f14b80b8080fe9d72e3215` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:28:45.004899 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=883aca80f1f14b80b8080fe9d72e3215)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:28:45.007412 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:28:45.116449 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '883aca80f1f14b80b8080fe9d72e3215', 'request_id': 'req-e697e2e6-352d-4045-b150-a276f22fc5ac'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:45.189135 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '883aca80f1f14b80b8080fe9d72e3215', 'request_id': 'req-e697e2e6-352d-4045-b150-a276f22fc5ac'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:45.196294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '883aca80f1f14b80b8080fe9d72e3215', 'request_id': 'req-e697e2e6-352d-4045-b150-a276f22fc5ac'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:45.207677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] The token cache is being invalidate because project 883aca80f1f14b80b8080fe9d72e3215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:28:45.207961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:45.210432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e697e2e6-352d-4045-b150-a276f22fc5ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:28:45.210702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1310/2616] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:28:44 2022] DELETE /identity/v3/projects/883aca80f1f14b80b8080fe9d72e3215 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:28:54.634083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:28:55.208946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:29:07.954068 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:07.955510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:07.969367 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cd086-823c-423e-8044-cbe63facac3a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:08.026007 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:08.026860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:08.027138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:08.027413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] PATH_INFO: `/v3/users/1a3814e4962f4e2fb7956b7a19af75d2` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:08.028564 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a3814e4962f4e2fb7956b7a19af75d2)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:08.031152 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:08.274708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a3814e4962f4e2fb7956b7a19af75d2'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:08.369631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a3814e4962f4e2fb7956b7a19af75d2'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:08.377681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a3814e4962f4e2fb7956b7a19af75d2'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:08.390107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a3814e4962f4e2fb7956b7a19af75d2'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:08.390762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a3814e4962f4e2fb7956b7a19af75d2'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:08.396593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1307/2617] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:29:07 2022] DELETE /identity/v3/users/1a3814e4962f4e2fb7956b7a19af75d2 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:29:08.401128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bc54f-9ec0-41e6-81a1-6231509595c9 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:08.401678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bc54f-9ec0-41e6-81a1-6231509595c9 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:08.401821 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bc54f-9ec0-41e6-81a1-6231509595c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:08.431171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b21bc54f-9ec0-41e6-81a1-6231509595c9 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:29:08.432373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b21bc54f-9ec0-41e6-81a1-6231509595c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:08.499567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1311/2618] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:29:08.509976 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:08.511441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:08.580303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c007bb-97c4-48c3-b62e-c261a47df2b1 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:08.584081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a3468-a8a5-4d76-99a3-054a5a144909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:08.584682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3468-a8a5-4d76-99a3-054a5a144909 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:08.585005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3468-a8a5-4d76-99a3-054a5a144909 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:08.585252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3468-a8a5-4d76-99a3-054a5a144909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:08.595139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a3468-a8a5-4d76-99a3-054a5a144909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:08.596712 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a3468-a8a5-4d76-99a3-054a5a144909 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:08.606891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1308/2619] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:29:09.114931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bc54f-9ec0-41e6-81a1-6231509595c9 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:09.116431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b21bc54f-9ec0-41e6-81a1-6231509595c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:09.129629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bc54f-9ec0-41e6-81a1-6231509595c9 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:09.134866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:09.135498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:09.135774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:09.136045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] PATH_INFO: `/v3/projects/95580e3f037c419d8eba0dbaace0348e` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:09.137191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=95580e3f037c419d8eba0dbaace0348e)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:09.139152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:09.272836 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95580e3f037c419d8eba0dbaace0348e', 'request_id': 'req-42943e9a-4363-43c1-a5ae-05946424aa5e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:09.323582 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95580e3f037c419d8eba0dbaace0348e', 'request_id': 'req-42943e9a-4363-43c1-a5ae-05946424aa5e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:09.331247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95580e3f037c419d8eba0dbaace0348e', 'request_id': 'req-42943e9a-4363-43c1-a5ae-05946424aa5e'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:09.341886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] The token cache is being invalidate because project 95580e3f037c419d8eba0dbaace0348e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:29:09.347585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:09.347585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-42943e9a-4363-43c1-a5ae-05946424aa5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:09.347585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1312/2620] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:29:09 2022] DELETE /identity/v3/projects/95580e3f037c419d8eba0dbaace0348e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:29:09.349784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26471b3-f9fe-41c2-8efa-bbcf8a5f2c1c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:09.350269 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26471b3-f9fe-41c2-8efa-bbcf8a5f2c1c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:09.350405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26471b3-f9fe-41c2-8efa-bbcf8a5f2c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:09.384835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:29:09.390698 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-e26471b3-f9fe-41c2-8efa-bbcf8a5f2c1c None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:29:09.394923 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e26471b3-f9fe-41c2-8efa-bbcf8a5f2c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:09.453462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1309/2621] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:29:09.461539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c6a22-8dc6-4acc-ae03-8e35ee5ac50b None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:09.462128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c6a22-8dc6-4acc-ae03-8e35ee5ac50b None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:09.462128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c6a22-8dc6-4acc-ae03-8e35ee5ac50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:09.483235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-281c6a22-8dc6-4acc-ae03-8e35ee5ac50b None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:29:09.485122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-281c6a22-8dc6-4acc-ae03-8e35ee5ac50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:09.493248 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1313/2622] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:29:09.497530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26471b3-f9fe-41c2-8efa-bbcf8a5f2c1c None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:09.507665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26471b3-f9fe-41c2-8efa-bbcf8a5f2c1c None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:09.514849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe10306-473f-412a-8c77-8c85471849a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:09.515482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe10306-473f-412a-8c77-8c85471849a5 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:09.515760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe10306-473f-412a-8c77-8c85471849a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:09.516031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe10306-473f-412a-8c77-8c85471849a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:09.516676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe10306-473f-412a-8c77-8c85471849a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:29:09.516975 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe10306-473f-412a-8c77-8c85471849a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:09.518604 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe10306-473f-412a-8c77-8c85471849a5 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:09.531492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1310/2623] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:29:09.535870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c6a22-8dc6-4acc-ae03-8e35ee5ac50b None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:09.545158 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c6a22-8dc6-4acc-ae03-8e35ee5ac50b None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:09.550998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f374d21-0afb-4d6f-89b4-08ff54717108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:09.551284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f374d21-0afb-4d6f-89b4-08ff54717108 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:09.552068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f374d21-0afb-4d6f-89b4-08ff54717108 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:09.552068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f374d21-0afb-4d6f-89b4-08ff54717108 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:09.552581 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f374d21-0afb-4d6f-89b4-08ff54717108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:09.554806 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f374d21-0afb-4d6f-89b4-08ff54717108 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:09.646718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1314/2624] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:29:09 2022] POST /identity/v3/projects => generated 391 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:29:09.650859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe10306-473f-412a-8c77-8c85471849a5 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:09.661389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe10306-473f-412a-8c77-8c85471849a5 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:09.723129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b50b9-b575-4c8a-804d-e7d9626c16e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:09.723725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b50b9-b575-4c8a-804d-e7d9626c16e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:09.723996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b50b9-b575-4c8a-804d-e7d9626c16e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:09.724262 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b50b9-b575-4c8a-804d-e7d9626c16e4 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:09.724890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b50b9-b575-4c8a-804d-e7d9626c16e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:09.726816 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b50b9-b575-4c8a-804d-e7d9626c16e4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:09.799420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1311/2625] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:29:09 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:29:09.804706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f374d21-0afb-4d6f-89b4-08ff54717108 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:09.820622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f374d21-0afb-4d6f-89b4-08ff54717108 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:09.829353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b027f46-7fcd-4723-bc08-1051706691f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:09.830215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b027f46-7fcd-4723-bc08-1051706691f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:09.830486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b027f46-7fcd-4723-bc08-1051706691f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:09.830750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b027f46-7fcd-4723-bc08-1051706691f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:09.831391 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b027f46-7fcd-4723-bc08-1051706691f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:09.832962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b027f46-7fcd-4723-bc08-1051706691f8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:09.844823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1315/2626] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:29:09.848995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b50b9-b575-4c8a-804d-e7d9626c16e4 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:09.859153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b50b9-b575-4c8a-804d-e7d9626c16e4 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:09.863810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98838e61-a8e0-47d9-93ff-1d65242c2af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:09.864389 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98838e61-a8e0-47d9-93ff-1d65242c2af8 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:09.864660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98838e61-a8e0-47d9-93ff-1d65242c2af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:09.864927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98838e61-a8e0-47d9-93ff-1d65242c2af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:09.865569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98838e61-a8e0-47d9-93ff-1d65242c2af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:09.867192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98838e61-a8e0-47d9-93ff-1d65242c2af8 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:09.878058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1312/2627] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:29:09.882512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b027f46-7fcd-4723-bc08-1051706691f8 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:09.893853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b027f46-7fcd-4723-bc08-1051706691f8 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:09.899251 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a7fe2-1eae-4f1a-aa22-787add84e508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:09.899839 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7fe2-1eae-4f1a-aa22-787add84e508 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:09.900115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7fe2-1eae-4f1a-aa22-787add84e508 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:09.900385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a7fe2-1eae-4f1a-aa22-787add84e508 admin admin] PATH_INFO: `/v3/projects/d17b8821e7b84bb88317a8a743c5284c/users/861c57ac921747e7a63220043b96786b/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:09.919833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a7fe2-1eae-4f1a-aa22-787add84e508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d17b8821e7b84bb88317a8a743c5284c, user_id=861c57ac921747e7a63220043b96786b, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:09.922518 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a7fe2-1eae-4f1a-aa22-787add84e508 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:10.034020 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1316/2628] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:29:09 2022] PUT /identity/v3/projects/d17b8821e7b84bb88317a8a743c5284c/users/861c57ac921747e7a63220043b96786b/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:29:10.044707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb5065-4dfc-4007-8951-7ea131f870c9 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:10.044991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb5065-4dfc-4007-8951-7ea131f870c9 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:10.050932 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb5065-4dfc-4007-8951-7ea131f870c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:10.076100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-52eb5065-4dfc-4007-8951-7ea131f870c9 None None] MFA Rules not processed for user `861c57ac921747e7a63220043b96786b`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:29:10.077334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-52eb5065-4dfc-4007-8951-7ea131f870c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:10.149575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1313/2629] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:29:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:29:10.160303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a7fe2-1eae-4f1a-aa22-787add84e508 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:10.162153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-901a7fe2-1eae-4f1a-aa22-787add84e508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:10.178492 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:29:10.222791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a7fe2-1eae-4f1a-aa22-787add84e508 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:10.226818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b15bc3-642d-4421-a06e-88aa6cd9bfa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:10.227489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b15bc3-642d-4421-a06e-88aa6cd9bfa5 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:10.227773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b15bc3-642d-4421-a06e-88aa6cd9bfa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:10.228034 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b15bc3-642d-4421-a06e-88aa6cd9bfa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:10.238006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b15bc3-642d-4421-a06e-88aa6cd9bfa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:10.239609 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b15bc3-642d-4421-a06e-88aa6cd9bfa5 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:10.249106 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1317/2630] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:29:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:29:20.146418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:29:20.397838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=20 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:29:35.419277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb5065-4dfc-4007-8951-7ea131f870c9 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:35.482364 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb5065-4dfc-4007-8951-7ea131f870c9 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:35.487537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:35.488159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:35.488440 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:35.488715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] PATH_INFO: `/v3/users/861c57ac921747e7a63220043b96786b` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:35.489834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=861c57ac921747e7a63220043b96786b)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:35.491769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:35.678546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '861c57ac921747e7a63220043b96786b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:35.773361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '861c57ac921747e7a63220043b96786b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:35.780699 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '861c57ac921747e7a63220043b96786b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:35.795340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '861c57ac921747e7a63220043b96786b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:35.801578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '861c57ac921747e7a63220043b96786b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:35.804045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1314/2631] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:29:35 2022] DELETE /identity/v3/users/861c57ac921747e7a63220043b96786b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:29:35.808788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e34a-0017-4746-b674-86b7a59290cd None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:35.809327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e34a-0017-4746-b674-86b7a59290cd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:35.809327 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7e34a-0017-4746-b674-86b7a59290cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:35.827998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-d1b7e34a-0017-4746-b674-86b7a59290cd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:29:35.829313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d1b7e34a-0017-4746-b674-86b7a59290cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:35.887304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1318/2632] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:29:35.897788 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:35.899274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:35.955549 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0eb03-4356-4bfe-ae48-f2d0425c8d4d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:35.959056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629e456-8310-44cc-8cd2-100f27da0d7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:35.959642 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629e456-8310-44cc-8cd2-100f27da0d7a service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:35.959917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629e456-8310-44cc-8cd2-100f27da0d7a service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:35.960187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629e456-8310-44cc-8cd2-100f27da0d7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:35.969036 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629e456-8310-44cc-8cd2-100f27da0d7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:35.971089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629e456-8310-44cc-8cd2-100f27da0d7a service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:35.980376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1315/2633] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:29:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:29:36.406766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7e34a-0017-4746-b674-86b7a59290cd None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:29:36.412394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-d1b7e34a-0017-4746-b674-86b7a59290cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:29:36.433400 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7e34a-0017-4746-b674-86b7a59290cd None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:29:36.439103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:29:36.439729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:29:36.440007 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:29:36.440277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] PATH_INFO: `/v3/projects/d17b8821e7b84bb88317a8a743c5284c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:29:36.441393 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d17b8821e7b84bb88317a8a743c5284c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:29:36.443302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:29:36.594701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd17b8821e7b84bb88317a8a743c5284c', 'request_id': 'req-77125a90-731e-4207-adac-2a8a4a340657'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:36.682701 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd17b8821e7b84bb88317a8a743c5284c', 'request_id': 'req-77125a90-731e-4207-adac-2a8a4a340657'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:36.690403 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd17b8821e7b84bb88317a8a743c5284c', 'request_id': 'req-77125a90-731e-4207-adac-2a8a4a340657'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:36.701767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] The token cache is being invalidate because project d17b8821e7b84bb88317a8a743c5284c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:29:36.702225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:36.702858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:29:36.704524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1319/2634] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:29:36 2022] DELETE /identity/v3/projects/d17b8821e7b84bb88317a8a743c5284c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:29:45.976227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:29:46.698773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:02.557089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629e456-8310-44cc-8cd2-100f27da0d7a service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:02.558520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-f629e456-8310-44cc-8cd2-100f27da0d7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:02.569834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629e456-8310-44cc-8cd2-100f27da0d7a service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:02.618607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786064a6-64fc-49c1-9618-9bd14f3ba290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:02.619201 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786064a6-64fc-49c1-9618-9bd14f3ba290 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:02.619475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786064a6-64fc-49c1-9618-9bd14f3ba290 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:02.619747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786064a6-64fc-49c1-9618-9bd14f3ba290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:02.621595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-786064a6-64fc-49c1-9618-9bd14f3ba290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:02.696916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786064a6-64fc-49c1-9618-9bd14f3ba290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:02.698476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786064a6-64fc-49c1-9618-9bd14f3ba290 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:02.708378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1316/2635] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:30:06.119497 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:06.121722 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:06.136343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77125a90-731e-4207-adac-2a8a4a340657 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:06.144619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:06.145861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:06.146167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:06.146439 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] PATH_INFO: `/v3/users/6b7457ab864e4df6b10a195f8500c14c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:06.160862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b7457ab864e4df6b10a195f8500c14c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:06.162807 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:06.364377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b7457ab864e4df6b10a195f8500c14c'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:06.434346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b7457ab864e4df6b10a195f8500c14c'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:06.449796 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b7457ab864e4df6b10a195f8500c14c'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:06.450416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b7457ab864e4df6b10a195f8500c14c'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:06.451022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3dfed089-3333-42eb-af8b-b82e1bb7a54d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b7457ab864e4df6b10a195f8500c14c'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:06.464051 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1320/2636] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:30:06 2022] DELETE /identity/v3/users/6b7457ab864e4df6b10a195f8500c14c => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:06.775811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786064a6-64fc-49c1-9618-9bd14f3ba290 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:06.784993 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786064a6-64fc-49c1-9618-9bd14f3ba290 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:06.849910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:06.850561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:06.850834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:06.851103 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] PATH_INFO: `/v3/projects/06c88ae5d5914faba8ff52380a8fcafa` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:06.854562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c88ae5d5914faba8ff52380a8fcafa)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:06.856664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:06.968428 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c88ae5d5914faba8ff52380a8fcafa', 'request_id': 'req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:06.975052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:07.052267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c88ae5d5914faba8ff52380a8fcafa', 'request_id': 'req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:07.059252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c88ae5d5914faba8ff52380a8fcafa', 'request_id': 'req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:07.070836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] The token cache is being invalidate because project 06c88ae5d5914faba8ff52380a8fcafa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:30:07.071121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:07.071785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0ae6ec4b-1b70-4f37-9e9b-77f80752fcee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:07.073309 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1317/2637] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:30:06 2022] DELETE /identity/v3/projects/06c88ae5d5914faba8ff52380a8fcafa => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:07.078402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e15489-6c26-4e67-a32a-b4124093a608 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:07.080500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e15489-6c26-4e67-a32a-b4124093a608 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:07.080500 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e15489-6c26-4e67-a32a-b4124093a608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:07.100756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-02e15489-6c26-4e67-a32a-b4124093a608 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:07.102075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-02e15489-6c26-4e67-a32a-b4124093a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:07.175727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1321/2638] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:07.184802 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c952c-753e-42d0-b790-982b70e2c874 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:07.185076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c952c-753e-42d0-b790-982b70e2c874 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:07.185338 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c952c-753e-42d0-b790-982b70e2c874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:07.205327 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c65c952c-753e-42d0-b790-982b70e2c874 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:07.206557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c65c952c-753e-42d0-b790-982b70e2c874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:07.215029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1318/2639] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:30:07.219561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e15489-6c26-4e67-a32a-b4124093a608 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:07.229299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e15489-6c26-4e67-a32a-b4124093a608 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:07.234023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f2e0a-27d3-4f66-a64d-baf6b5c1f585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:07.234626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f2e0a-27d3-4f66-a64d-baf6b5c1f585 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:07.234896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f2e0a-27d3-4f66-a64d-baf6b5c1f585 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:07.235186 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f2e0a-27d3-4f66-a64d-baf6b5c1f585 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:07.235831 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f2e0a-27d3-4f66-a64d-baf6b5c1f585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:30:07.236130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f2e0a-27d3-4f66-a64d-baf6b5c1f585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:07.237697 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f2e0a-27d3-4f66-a64d-baf6b5c1f585 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:07.250834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1322/2640] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:30:07.259115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c952c-753e-42d0-b790-982b70e2c874 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:07.268502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c952c-753e-42d0-b790-982b70e2c874 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:07.273243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d17198-0d45-41c9-a20f-36bea98b08a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:07.273868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d17198-0d45-41c9-a20f-36bea98b08a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:07.274185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d17198-0d45-41c9-a20f-36bea98b08a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:07.274451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d17198-0d45-41c9-a20f-36bea98b08a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:07.275082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d17198-0d45-41c9-a20f-36bea98b08a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:07.276968 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d17198-0d45-41c9-a20f-36bea98b08a0 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:07.368942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1319/2641] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:30:07 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:07.373129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f2e0a-27d3-4f66-a64d-baf6b5c1f585 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:07.384016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f2e0a-27d3-4f66-a64d-baf6b5c1f585 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:07.437147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:07.452645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9b3b7-0318-491d-8416-e796de46d5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:07.453243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9b3b7-0318-491d-8416-e796de46d5df admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:07.453523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9b3b7-0318-491d-8416-e796de46d5df admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:07.453886 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9b3b7-0318-491d-8416-e796de46d5df admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:07.454580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9b3b7-0318-491d-8416-e796de46d5df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:07.456543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a9b3b7-0318-491d-8416-e796de46d5df admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:07.543885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1323/2642] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:30:07 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:07.552168 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d17198-0d45-41c9-a20f-36bea98b08a0 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:07.563313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d17198-0d45-41c9-a20f-36bea98b08a0 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:07.568530 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd44bf6-6dea-43a1-ab40-6c8471d573f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:07.569120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd44bf6-6dea-43a1-ab40-6c8471d573f6 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:07.569396 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd44bf6-6dea-43a1-ab40-6c8471d573f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:07.569664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd44bf6-6dea-43a1-ab40-6c8471d573f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:07.570332 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd44bf6-6dea-43a1-ab40-6c8471d573f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:07.571892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd44bf6-6dea-43a1-ab40-6c8471d573f6 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:07.581674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1320/2643] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:07.585987 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9b3b7-0318-491d-8416-e796de46d5df admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:07.596343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9b3b7-0318-491d-8416-e796de46d5df admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:07.601467 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a99fcd-03a2-4c67-8d11-af40f0f1978b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:07.602094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a99fcd-03a2-4c67-8d11-af40f0f1978b admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:07.602398 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a99fcd-03a2-4c67-8d11-af40f0f1978b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:07.602664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a99fcd-03a2-4c67-8d11-af40f0f1978b admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:07.603301 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a99fcd-03a2-4c67-8d11-af40f0f1978b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:07.604854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a99fcd-03a2-4c67-8d11-af40f0f1978b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:07.615842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1324/2644] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:07.620083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd44bf6-6dea-43a1-ab40-6c8471d573f6 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:07.629927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd44bf6-6dea-43a1-ab40-6c8471d573f6 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:07.634784 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47d92e-d409-4a3c-94db-2fd30adc7c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:07.635412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47d92e-d409-4a3c-94db-2fd30adc7c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:07.635684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47d92e-d409-4a3c-94db-2fd30adc7c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:07.635949 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47d92e-d409-4a3c-94db-2fd30adc7c4f admin admin] PATH_INFO: `/v3/projects/5f14d91e296a43eba1741d87e5f1bb85/users/3b0ba9438109496a9d08a34a6b1707f4/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:07.650222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47d92e-d409-4a3c-94db-2fd30adc7c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f14d91e296a43eba1741d87e5f1bb85, user_id=3b0ba9438109496a9d08a34a6b1707f4, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:07.652839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47d92e-d409-4a3c-94db-2fd30adc7c4f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:07.709184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1321/2645] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:30:07 2022] PUT /identity/v3/projects/5f14d91e296a43eba1741d87e5f1bb85/users/3b0ba9438109496a9d08a34a6b1707f4/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:07.719422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c592a5-8463-4df2-b13e-481bf13a8166 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:07.719703 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c592a5-8463-4df2-b13e-481bf13a8166 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:07.719973 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c592a5-8463-4df2-b13e-481bf13a8166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:07.749542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-42c592a5-8463-4df2-b13e-481bf13a8166 None None] MFA Rules not processed for user `3b0ba9438109496a9d08a34a6b1707f4`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:07.750879 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-42c592a5-8463-4df2-b13e-481bf13a8166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:07.819979 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1325/2646] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:07.831317 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47d92e-d409-4a3c-94db-2fd30adc7c4f admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:07.833081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7e47d92e-d409-4a3c-94db-2fd30adc7c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:07.906901 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47d92e-d409-4a3c-94db-2fd30adc7c4f admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:07.916653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6903706-243a-47d3-9e29-c32fed91cab8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:07.917312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6903706-243a-47d3-9e29-c32fed91cab8 service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:07.917561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6903706-243a-47d3-9e29-c32fed91cab8 service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:07.917832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6903706-243a-47d3-9e29-c32fed91cab8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:07.927597 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6903706-243a-47d3-9e29-c32fed91cab8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:07.929204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6903706-243a-47d3-9e29-c32fed91cab8 service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:07.940412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1322/2647] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:30:15.295174 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c592a5-8463-4df2-b13e-481bf13a8166 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:15.296580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-42c592a5-8463-4df2-b13e-481bf13a8166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:15.365377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c592a5-8463-4df2-b13e-481bf13a8166 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:15.368938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bfad8-d2bb-4dea-a932-ad2de243df09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:15.376943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bfad8-d2bb-4dea-a932-ad2de243df09 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:15.376943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bfad8-d2bb-4dea-a932-ad2de243df09 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:15.376943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bfad8-d2bb-4dea-a932-ad2de243df09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:15.376943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-404bfad8-d2bb-4dea-a932-ad2de243df09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:15.437675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:15.454895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bfad8-d2bb-4dea-a932-ad2de243df09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:15.456312 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bfad8-d2bb-4dea-a932-ad2de243df09 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:15.476743 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1326/2648] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:30:17.936612 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=3,keystone:SELECT=67,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:18.248108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6903706-243a-47d3-9e29-c32fed91cab8 service nova] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:18.318687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6903706-243a-47d3-9e29-c32fed91cab8 service nova] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:18.323467 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:18.324178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:18.324462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:18.325046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] PATH_INFO: `/v3/users/3b0ba9438109496a9d08a34a6b1707f4` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:18.326537 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b0ba9438109496a9d08a34a6b1707f4)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:18.329205 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:18.530496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b0ba9438109496a9d08a34a6b1707f4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:18.588503 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b0ba9438109496a9d08a34a6b1707f4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:18.596488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b0ba9438109496a9d08a34a6b1707f4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:18.610809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b0ba9438109496a9d08a34a6b1707f4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:18.611595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b0ba9438109496a9d08a34a6b1707f4'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:18.617826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1323/2649] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:30:18 2022] DELETE /identity/v3/users/3b0ba9438109496a9d08a34a6b1707f4 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:18.622378 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f863333-ab8f-4e7f-96fc-a6c9c4bc527e None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:18.622933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f863333-ab8f-4e7f-96fc-a6c9c4bc527e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:18.622933 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f863333-ab8f-4e7f-96fc-a6c9c4bc527e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:18.641752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-3f863333-ab8f-4e7f-96fc-a6c9c4bc527e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:18.643112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3f863333-ab8f-4e7f-96fc-a6c9c4bc527e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:18.708677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1327/2650] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:18.724339 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:18.727217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:18.790702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601385ed-0e2b-4f72-9996-c797a7e19460 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:18.794668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085eb5be-8948-4998-a2c3-4de575a57158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:18.795286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eb5be-8948-4998-a2c3-4de575a57158 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:18.795563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eb5be-8948-4998-a2c3-4de575a57158 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:18.795835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eb5be-8948-4998-a2c3-4de575a57158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:18.805685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085eb5be-8948-4998-a2c3-4de575a57158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:18.807153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085eb5be-8948-4998-a2c3-4de575a57158 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:18.817109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1324/2651] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:30:19.277046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f863333-ab8f-4e7f-96fc-a6c9c4bc527e None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:19.278667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3f863333-ab8f-4e7f-96fc-a6c9c4bc527e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:19.294707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f863333-ab8f-4e7f-96fc-a6c9c4bc527e None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:19.300154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:19.300797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:19.301072 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:19.301346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] PATH_INFO: `/v3/projects/5f14d91e296a43eba1741d87e5f1bb85` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:19.302615 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f14d91e296a43eba1741d87e5f1bb85)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:19.305089 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:19.438515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f14d91e296a43eba1741d87e5f1bb85', 'request_id': 'req-89e02058-26bb-4413-b1e7-35fe9c353b8f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:19.553751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f14d91e296a43eba1741d87e5f1bb85', 'request_id': 'req-89e02058-26bb-4413-b1e7-35fe9c353b8f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:19.561387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f14d91e296a43eba1741d87e5f1bb85', 'request_id': 'req-89e02058-26bb-4413-b1e7-35fe9c353b8f'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:19.572472 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] The token cache is being invalidate because project 5f14d91e296a43eba1741d87e5f1bb85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:30:19.572761 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:19.573416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-89e02058-26bb-4413-b1e7-35fe9c353b8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:19.574931 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1328/2652] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:30:19 2022] DELETE /identity/v3/projects/5f14d91e296a43eba1741d87e5f1bb85 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:19.581141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d74d3-78d5-46e5-96ae-421ccf9fd4cc None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:19.581787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d74d3-78d5-46e5-96ae-421ccf9fd4cc None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:19.581787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d74d3-78d5-46e5-96ae-421ccf9fd4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:19.603240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ac1d74d3-78d5-46e5-96ae-421ccf9fd4cc None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:19.604534 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ac1d74d3-78d5-46e5-96ae-421ccf9fd4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:19.614757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:19.676127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1325/2653] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:19.684858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51656-1085-43ca-ae63-11d05686757f None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:19.685628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51656-1085-43ca-ae63-11d05686757f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:19.685628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd51656-1085-43ca-ae63-11d05686757f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:19.705812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2cd51656-1085-43ca-ae63-11d05686757f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:19.707076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2cd51656-1085-43ca-ae63-11d05686757f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:19.715338 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1329/2654] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:19.719941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d74d3-78d5-46e5-96ae-421ccf9fd4cc None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:19.730896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d74d3-78d5-46e5-96ae-421ccf9fd4cc None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:19.735892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe18fec-cabc-42f0-8103-a9ef0ace47e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:19.736516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe18fec-cabc-42f0-8103-a9ef0ace47e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:19.736791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe18fec-cabc-42f0-8103-a9ef0ace47e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:19.737058 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe18fec-cabc-42f0-8103-a9ef0ace47e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:19.737726 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe18fec-cabc-42f0-8103-a9ef0ace47e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:30:19.738053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe18fec-cabc-42f0-8103-a9ef0ace47e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:19.739619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe18fec-cabc-42f0-8103-a9ef0ace47e7 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:19.753477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1326/2655] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:30:19.757997 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd51656-1085-43ca-ae63-11d05686757f None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:19.767303 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd51656-1085-43ca-ae63-11d05686757f None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:19.772489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535f029-6010-4648-a521-c9c8d2dde3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:19.773090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f029-6010-4648-a521-c9c8d2dde3ae admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:19.773361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f029-6010-4648-a521-c9c8d2dde3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:19.773623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535f029-6010-4648-a521-c9c8d2dde3ae admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:19.774275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535f029-6010-4648-a521-c9c8d2dde3ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:19.777074 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535f029-6010-4648-a521-c9c8d2dde3ae admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:19.816926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe18fec-cabc-42f0-8103-a9ef0ace47e7 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:19.818728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bbe18fec-cabc-42f0-8103-a9ef0ace47e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:19.885512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1330/2656] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:30:19 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:19.889613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535f029-6010-4648-a521-c9c8d2dde3ae admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:19.906350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe18fec-cabc-42f0-8103-a9ef0ace47e7 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:19.925800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:19.959519 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535f029-6010-4648-a521-c9c8d2dde3ae admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:19.965727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fea770-6271-4f64-9629-feb4c6c4b05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:19.966364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fea770-6271-4f64-9629-feb4c6c4b05c admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:19.966638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fea770-6271-4f64-9629-feb4c6c4b05c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:19.966932 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fea770-6271-4f64-9629-feb4c6c4b05c admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:19.967553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fea770-6271-4f64-9629-feb4c6c4b05c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:19.969495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fea770-6271-4f64-9629-feb4c6c4b05c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:19.984792 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004ae83-c7c9-4735-ab2b-c5b41cbd4a1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:19.985464 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ae83-c7c9-4735-ab2b-c5b41cbd4a1f service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:19.986077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ae83-c7c9-4735-ab2b-c5b41cbd4a1f service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:19.986578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ae83-c7c9-4735-ab2b-c5b41cbd4a1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:19.992911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6004ae83-c7c9-4735-ab2b-c5b41cbd4a1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:20.000652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004ae83-c7c9-4735-ab2b-c5b41cbd4a1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:20.002293 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004ae83-c7c9-4735-ab2b-c5b41cbd4a1f service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:20.013021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1327/2657] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:30:20.104298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1331/2658] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:30:19 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:20.109490 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004ae83-c7c9-4735-ab2b-c5b41cbd4a1f service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:20.122461 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004ae83-c7c9-4735-ab2b-c5b41cbd4a1f service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:20.130189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209515c-86b7-4968-8df6-d476e6e1540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:20.130189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209515c-86b7-4968-8df6-d476e6e1540f admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:20.130189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209515c-86b7-4968-8df6-d476e6e1540f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:20.130189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209515c-86b7-4968-8df6-d476e6e1540f admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:20.130632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209515c-86b7-4968-8df6-d476e6e1540f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:20.131815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209515c-86b7-4968-8df6-d476e6e1540f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:20.142907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1328/2659] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:20.150845 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fea770-6271-4f64-9629-feb4c6c4b05c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:20.163406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fea770-6271-4f64-9629-feb4c6c4b05c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:20.168302 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71a996-e194-4bf5-a614-c8fe8eed948e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:20.168898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71a996-e194-4bf5-a614-c8fe8eed948e admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:20.169212 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71a996-e194-4bf5-a614-c8fe8eed948e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:20.169434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71a996-e194-4bf5-a614-c8fe8eed948e admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:20.170129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71a996-e194-4bf5-a614-c8fe8eed948e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:20.172713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71a996-e194-4bf5-a614-c8fe8eed948e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:20.183687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1332/2660] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:20.193459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209515c-86b7-4968-8df6-d476e6e1540f admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:20.205190 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209515c-86b7-4968-8df6-d476e6e1540f admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:20.210517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd186c85-2990-4523-95db-7e355f47c584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:20.211174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd186c85-2990-4523-95db-7e355f47c584 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:20.211444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd186c85-2990-4523-95db-7e355f47c584 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:20.211710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd186c85-2990-4523-95db-7e355f47c584 admin admin] PATH_INFO: `/v3/projects/3ce4024e37a344e69420f69bbd7031a8/users/f4992507173a42a1a4c467941023c403/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:20.356031 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=100 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:20.359408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd186c85-2990-4523-95db-7e355f47c584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce4024e37a344e69420f69bbd7031a8, user_id=f4992507173a42a1a4c467941023c403, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:20.362343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd186c85-2990-4523-95db-7e355f47c584 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:20.446335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1329/2661] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:30:20 2022] PUT /identity/v3/projects/3ce4024e37a344e69420f69bbd7031a8/users/f4992507173a42a1a4c467941023c403/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:20.458516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa836-55e8-418f-b1e6-3b8e64474cea None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:20.458799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa836-55e8-418f-b1e6-3b8e64474cea None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:20.459752 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faa836-55e8-418f-b1e6-3b8e64474cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:20.487395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-71faa836-55e8-418f-b1e6-3b8e64474cea None None] MFA Rules not processed for user `f4992507173a42a1a4c467941023c403`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:20.488695 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-71faa836-55e8-418f-b1e6-3b8e64474cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:20.557892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1333/2662] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:20.562581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd186c85-2990-4523-95db-7e355f47c584 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:20.627956 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd186c85-2990-4523-95db-7e355f47c584 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:20.633318 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10997e85-d062-4c8f-84f8-ed0cb411f0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:20.634186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997e85-d062-4c8f-84f8-ed0cb411f0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:20.634548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997e85-d062-4c8f-84f8-ed0cb411f0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:20.634846 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10997e85-d062-4c8f-84f8-ed0cb411f0a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:20.635429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10997e85-d062-4c8f-84f8-ed0cb411f0a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:20.637295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10997e85-d062-4c8f-84f8-ed0cb411f0a4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:20.796319 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1330/2663] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:30:20 2022] POST /identity/v3/projects => generated 365 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:20.800299 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faa836-55e8-418f-b1e6-3b8e64474cea None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:20.813124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faa836-55e8-418f-b1e6-3b8e64474cea None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:20.876896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caadcf4-1924-46d5-ba4c-d5bfdfcc6e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:20.877478 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caadcf4-1924-46d5-ba4c-d5bfdfcc6e95 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:20.877749 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caadcf4-1924-46d5-ba4c-d5bfdfcc6e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:20.878127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caadcf4-1924-46d5-ba4c-d5bfdfcc6e95 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:20.878682 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caadcf4-1924-46d5-ba4c-d5bfdfcc6e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:20.880573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caadcf4-1924-46d5-ba4c-d5bfdfcc6e95 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:20.960512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1334/2664] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:30:20 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:20.964609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10997e85-d062-4c8f-84f8-ed0cb411f0a4 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:20.975357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10997e85-d062-4c8f-84f8-ed0cb411f0a4 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:20.980476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adcc2d-1504-4550-9d26-f7e2372e5871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:20.981039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adcc2d-1504-4550-9d26-f7e2372e5871 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:20.981310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adcc2d-1504-4550-9d26-f7e2372e5871 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:20.981573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adcc2d-1504-4550-9d26-f7e2372e5871 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:20.982236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adcc2d-1504-4550-9d26-f7e2372e5871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:20.983809 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adcc2d-1504-4550-9d26-f7e2372e5871 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:20.994999 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1331/2665] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:20.999253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caadcf4-1924-46d5-ba4c-d5bfdfcc6e95 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:21.010159 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caadcf4-1924-46d5-ba4c-d5bfdfcc6e95 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:21.014649 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8c0a8-cae9-4e2b-875b-4335df6d9d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:21.015282 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8c0a8-cae9-4e2b-875b-4335df6d9d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:21.015554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8c0a8-cae9-4e2b-875b-4335df6d9d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:21.015817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8c0a8-cae9-4e2b-875b-4335df6d9d8e admin admin] PATH_INFO: `/v3/projects/f26d954cd9294044a6e2439b5e734923/users/3e411734531e4c69aac4a99f931d0e5d/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:21.030935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8c0a8-cae9-4e2b-875b-4335df6d9d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26d954cd9294044a6e2439b5e734923, user_id=3e411734531e4c69aac4a99f931d0e5d, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:21.033521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8c0a8-cae9-4e2b-875b-4335df6d9d8e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:21.131904 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1335/2666] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:30:20 2022] PUT /identity/v3/projects/f26d954cd9294044a6e2439b5e734923/users/3e411734531e4c69aac4a99f931d0e5d/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:21.135814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adcc2d-1504-4550-9d26-f7e2372e5871 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:21.145709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adcc2d-1504-4550-9d26-f7e2372e5871 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:21.201636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461e1f4-1629-4ad8-bcd2-98480298f2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:21.202328 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461e1f4-1629-4ad8-bcd2-98480298f2ca admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:21.202598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461e1f4-1629-4ad8-bcd2-98480298f2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:21.202858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461e1f4-1629-4ad8-bcd2-98480298f2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:21.203481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461e1f4-1629-4ad8-bcd2-98480298f2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:21.205043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b461e1f4-1629-4ad8-bcd2-98480298f2ca admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:21.214609 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1332/2667] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:21.220138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8c0a8-cae9-4e2b-875b-4335df6d9d8e admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:21.226259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:21.231586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8c0a8-cae9-4e2b-875b-4335df6d9d8e admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:21.236392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d15907-7012-4969-af2e-c3f58010d515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:21.238079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d15907-7012-4969-af2e-c3f58010d515 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:21.238079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d15907-7012-4969-af2e-c3f58010d515 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:21.238079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d15907-7012-4969-af2e-c3f58010d515 admin admin] PATH_INFO: `/v3/projects/f26d954cd9294044a6e2439b5e734923/users/3e411734531e4c69aac4a99f931d0e5d/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:21.239603 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d15907-7012-4969-af2e-c3f58010d515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26d954cd9294044a6e2439b5e734923, user_id=3e411734531e4c69aac4a99f931d0e5d, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:21.242861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d15907-7012-4969-af2e-c3f58010d515 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:21.253185 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1336/2668] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:30:21 2022] PUT /identity/v3/projects/f26d954cd9294044a6e2439b5e734923/users/3e411734531e4c69aac4a99f931d0e5d/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:21.262982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058cdf6-4e9c-4040-8fc0-dd1cb69baab7 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:21.263264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058cdf6-4e9c-4040-8fc0-dd1cb69baab7 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:21.263533 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058cdf6-4e9c-4040-8fc0-dd1cb69baab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:21.291724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-6058cdf6-4e9c-4040-8fc0-dd1cb69baab7 None None] MFA Rules not processed for user `3e411734531e4c69aac4a99f931d0e5d`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:21.292926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6058cdf6-4e9c-4040-8fc0-dd1cb69baab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:21.341526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:21.362305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1333/2669] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:21.374132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d15907-7012-4969-af2e-c3f58010d515 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:21.375873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-00d15907-7012-4969-af2e-c3f58010d515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:21.434294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d15907-7012-4969-af2e-c3f58010d515 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:21.437916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35af9d-bb5d-4223-8f11-9e05a6fdc537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:21.438520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35af9d-bb5d-4223-8f11-9e05a6fdc537 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:21.438791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35af9d-bb5d-4223-8f11-9e05a6fdc537 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:21.439057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e35af9d-bb5d-4223-8f11-9e05a6fdc537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:21.448018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35af9d-bb5d-4223-8f11-9e05a6fdc537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:21.449457 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e35af9d-bb5d-4223-8f11-9e05a6fdc537 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:21.459528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1337/2670] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:30:21.683268 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058cdf6-4e9c-4040-8fc0-dd1cb69baab7 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:21.693296 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058cdf6-4e9c-4040-8fc0-dd1cb69baab7 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:21.696945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e38c9-acdc-4b5c-967e-514ccd13059b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:21.697583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e38c9-acdc-4b5c-967e-514ccd13059b service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:21.697860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e38c9-acdc-4b5c-967e-514ccd13059b service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:21.698157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e38c9-acdc-4b5c-967e-514ccd13059b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:21.755639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e38c9-acdc-4b5c-967e-514ccd13059b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:21.757570 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e38c9-acdc-4b5c-967e-514ccd13059b service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:21.766348 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1334/2671] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:30:31.454921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:31.762446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=34 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:35.952850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35af9d-bb5d-4223-8f11-9e05a6fdc537 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:36.014653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e35af9d-bb5d-4223-8f11-9e05a6fdc537 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:36.019238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:36.019847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:36.020120 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:36.020385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] PATH_INFO: `/v3/users/f4992507173a42a1a4c467941023c403` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:36.021490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4992507173a42a1a4c467941023c403)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:36.023382 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:36.200151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4992507173a42a1a4c467941023c403'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:36.273537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4992507173a42a1a4c467941023c403'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:36.286124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4992507173a42a1a4c467941023c403'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:36.286802 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4992507173a42a1a4c467941023c403'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:36.287411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4992507173a42a1a4c467941023c403'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:36.300797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1338/2672] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:30:35 2022] DELETE /identity/v3/users/f4992507173a42a1a4c467941023c403 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:36.305824 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ada1eb-ed67-4149-af4e-74ff6751f359 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:36.306361 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ada1eb-ed67-4149-af4e-74ff6751f359 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:36.306471 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ada1eb-ed67-4149-af4e-74ff6751f359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:36.324889 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c3ada1eb-ed67-4149-af4e-74ff6751f359 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:36.326112 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c3ada1eb-ed67-4149-af4e-74ff6751f359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:36.385432 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1335/2673] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:30:36.395329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:36.396894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:36.458664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee730a4-e45c-4b1e-a9ab-12b2516ec4ed admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:36.462921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735afa-39df-4d98-8986-f1e15f46e317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:36.463524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735afa-39df-4d98-8986-f1e15f46e317 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:36.464229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735afa-39df-4d98-8986-f1e15f46e317 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:36.464229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735afa-39df-4d98-8986-f1e15f46e317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:36.473975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46735afa-39df-4d98-8986-f1e15f46e317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:36.475394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46735afa-39df-4d98-8986-f1e15f46e317 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:36.486729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1339/2674] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:30:36.925071 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ada1eb-ed67-4149-af4e-74ff6751f359 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:36.926560 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c3ada1eb-ed67-4149-af4e-74ff6751f359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:36.940748 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ada1eb-ed67-4149-af4e-74ff6751f359 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:36.946928 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:36.947559 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:36.947966 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:36.948237 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] PATH_INFO: `/v3/projects/3ce4024e37a344e69420f69bbd7031a8` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:36.950655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce4024e37a344e69420f69bbd7031a8)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:36.959243 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:37.114172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce4024e37a344e69420f69bbd7031a8', 'request_id': 'req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:37.191912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce4024e37a344e69420f69bbd7031a8', 'request_id': 'req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:37.198658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce4024e37a344e69420f69bbd7031a8', 'request_id': 'req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:37.209166 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] The token cache is being invalidate because project 3ce4024e37a344e69420f69bbd7031a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:30:37.209501 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:37.210593 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:37.212132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1336/2675] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:30:36 2022] DELETE /identity/v3/projects/3ce4024e37a344e69420f69bbd7031a8 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:37.216352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735afa-39df-4d98-8986-f1e15f46e317 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:37.226820 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735afa-39df-4d98-8986-f1e15f46e317 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:37.245196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:37.282625 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:37.283235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:37.283510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:37.283774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] PATH_INFO: `/v3/users/3e411734531e4c69aac4a99f931d0e5d` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:37.284905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e411734531e4c69aac4a99f931d0e5d)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:37.286815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:37.559850 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e411734531e4c69aac4a99f931d0e5d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:37.656285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e411734531e4c69aac4a99f931d0e5d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:37.670869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e411734531e4c69aac4a99f931d0e5d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:37.671716 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e411734531e4c69aac4a99f931d0e5d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:37.672330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3b21f554-09fd-4cc5-83ef-4e00d841d989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e411734531e4c69aac4a99f931d0e5d'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:37.685134 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1340/2676] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:30:37 2022] DELETE /identity/v3/users/3e411734531e4c69aac4a99f931d0e5d => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:38.147755 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:38.149260 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:38.163459 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b77d6f-e86a-41c8-baed-4eaaf99edcf3 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:38.226574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:38.227855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:38.228119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:38.229147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] PATH_INFO: `/v3/projects/f26d954cd9294044a6e2439b5e734923` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:38.230548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26d954cd9294044a6e2439b5e734923)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:38.232462 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:38.314916 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:38.330732 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26d954cd9294044a6e2439b5e734923', 'request_id': 'req-9cbbabaf-05bb-4489-8e02-e9c380a63e97'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:38.410473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26d954cd9294044a6e2439b5e734923', 'request_id': 'req-9cbbabaf-05bb-4489-8e02-e9c380a63e97'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:38.417763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26d954cd9294044a6e2439b5e734923', 'request_id': 'req-9cbbabaf-05bb-4489-8e02-e9c380a63e97'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:38.428759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] The token cache is being invalidate because project f26d954cd9294044a6e2439b5e734923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:30:38.429046 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:38.429737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9cbbabaf-05bb-4489-8e02-e9c380a63e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:38.432165 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1337/2677] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:30:38 2022] DELETE /identity/v3/projects/f26d954cd9294044a6e2439b5e734923 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:38.437389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548b982-1bce-4d04-9e5c-ede96754bfe4 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:38.437971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548b982-1bce-4d04-9e5c-ede96754bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:38.437971 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548b982-1bce-4d04-9e5c-ede96754bfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:38.457187 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2548b982-1bce-4d04-9e5c-ede96754bfe4 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:38.458431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2548b982-1bce-4d04-9e5c-ede96754bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:38.518637 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1341/2678] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:38.527094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e520cf7-c48d-4f70-8baa-b6264ee70c1e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:38.527371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e520cf7-c48d-4f70-8baa-b6264ee70c1e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:38.527637 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e520cf7-c48d-4f70-8baa-b6264ee70c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:38.549302 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9e520cf7-c48d-4f70-8baa-b6264ee70c1e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:38.551489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9e520cf7-c48d-4f70-8baa-b6264ee70c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:38.562207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1338/2679] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:38.567102 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548b982-1bce-4d04-9e5c-ede96754bfe4 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:38.577129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548b982-1bce-4d04-9e5c-ede96754bfe4 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:38.582272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a0af1-0d4d-4bfa-8cdc-d7c05b6e3f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:38.582880 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0af1-0d4d-4bfa-8cdc-d7c05b6e3f7f admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:38.583154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0af1-0d4d-4bfa-8cdc-d7c05b6e3f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:38.583426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0af1-0d4d-4bfa-8cdc-d7c05b6e3f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:38.584070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a0af1-0d4d-4bfa-8cdc-d7c05b6e3f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:30:38.584369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a0af1-0d4d-4bfa-8cdc-d7c05b6e3f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:38.585945 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a0af1-0d4d-4bfa-8cdc-d7c05b6e3f7f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:38.599292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1342/2680] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:30:38.604506 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e520cf7-c48d-4f70-8baa-b6264ee70c1e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:38.614111 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e520cf7-c48d-4f70-8baa-b6264ee70c1e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:38.618892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1ace2-75c0-4f17-b658-ee9a30c3f4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:38.619479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1ace2-75c0-4f17-b658-ee9a30c3f4de admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:38.619780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1ace2-75c0-4f17-b658-ee9a30c3f4de admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:38.620049 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1ace2-75c0-4f17-b658-ee9a30c3f4de admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:38.620683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1ace2-75c0-4f17-b658-ee9a30c3f4de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:38.622601 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1ace2-75c0-4f17-b658-ee9a30c3f4de admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:38.715154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1339/2681] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:30:38 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:38.719595 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a0af1-0d4d-4bfa-8cdc-d7c05b6e3f7f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:38.730998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a0af1-0d4d-4bfa-8cdc-d7c05b6e3f7f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:38.760085 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:38.791528 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132b9f2-891f-4e6a-80e8-0fba10343c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:38.792160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132b9f2-891f-4e6a-80e8-0fba10343c67 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:38.792432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132b9f2-891f-4e6a-80e8-0fba10343c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:38.792772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132b9f2-891f-4e6a-80e8-0fba10343c67 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:38.793388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132b9f2-891f-4e6a-80e8-0fba10343c67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:38.795317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132b9f2-891f-4e6a-80e8-0fba10343c67 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:38.954589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1343/2682] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:30:38 2022] POST /identity/v3/users => generated 285 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:38.959896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1ace2-75c0-4f17-b658-ee9a30c3f4de admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:38.972095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1ace2-75c0-4f17-b658-ee9a30c3f4de admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:38.976239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e368537-1349-4dc9-a752-e9901798e8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:38.976841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e368537-1349-4dc9-a752-e9901798e8fc admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:38.977106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e368537-1349-4dc9-a752-e9901798e8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:38.977374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e368537-1349-4dc9-a752-e9901798e8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:38.978034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e368537-1349-4dc9-a752-e9901798e8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:38.979606 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e368537-1349-4dc9-a752-e9901798e8fc admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:38.989423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1340/2683] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:38.993783 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132b9f2-891f-4e6a-80e8-0fba10343c67 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:39.005567 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132b9f2-891f-4e6a-80e8-0fba10343c67 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:39.010870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be83a4-da3f-482e-b8fb-724219c54270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:39.011459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be83a4-da3f-482e-b8fb-724219c54270 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:39.011736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be83a4-da3f-482e-b8fb-724219c54270 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:39.012006 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be83a4-da3f-482e-b8fb-724219c54270 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:39.012689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be83a4-da3f-482e-b8fb-724219c54270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:39.014272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be83a4-da3f-482e-b8fb-724219c54270 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:39.025711 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1344/2684] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:39.030162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e368537-1349-4dc9-a752-e9901798e8fc admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:39.040013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e368537-1349-4dc9-a752-e9901798e8fc admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:39.044489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81d0d9-108a-4346-bd03-febb1871650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:39.045114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81d0d9-108a-4346-bd03-febb1871650c admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:39.045382 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81d0d9-108a-4346-bd03-febb1871650c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:39.045644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e81d0d9-108a-4346-bd03-febb1871650c admin admin] PATH_INFO: `/v3/projects/838cfd8867b3413a9b26ff40af8cc818/users/83cff6ace98f467ca689284af33688b0/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:39.060131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81d0d9-108a-4346-bd03-febb1871650c admin admin] RBAC: Authorizing `identity:create_grant(project_id=838cfd8867b3413a9b26ff40af8cc818, user_id=83cff6ace98f467ca689284af33688b0, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:39.062734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e81d0d9-108a-4346-bd03-febb1871650c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:39.136684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1341/2685] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:30:39 2022] PUT /identity/v3/projects/838cfd8867b3413a9b26ff40af8cc818/users/83cff6ace98f467ca689284af33688b0/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:39.147244 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad702eae-0681-4642-b332-41ef39413885 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:39.147523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad702eae-0681-4642-b332-41ef39413885 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:39.147807 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad702eae-0681-4642-b332-41ef39413885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:39.176157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ad702eae-0681-4642-b332-41ef39413885 None None] MFA Rules not processed for user `83cff6ace98f467ca689284af33688b0`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:39.177344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ad702eae-0681-4642-b332-41ef39413885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:39.244126 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1345/2686] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:39.248920 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81d0d9-108a-4346-bd03-febb1871650c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:39.309913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e81d0d9-108a-4346-bd03-febb1871650c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:39.314477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f4fce-084d-4967-9554-1bdf60be4172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:39.315057 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f4fce-084d-4967-9554-1bdf60be4172 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:39.315329 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f4fce-084d-4967-9554-1bdf60be4172 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:39.315594 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f4fce-084d-4967-9554-1bdf60be4172 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:39.316194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f4fce-084d-4967-9554-1bdf60be4172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:39.318069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f4fce-084d-4967-9554-1bdf60be4172 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:39.444182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1342/2687] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:30:39 2022] POST /identity/v3/projects => generated 369 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:39.449102 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad702eae-0681-4642-b332-41ef39413885 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:39.460223 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad702eae-0681-4642-b332-41ef39413885 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:39.521889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998aa3e-b1d4-4654-a5fc-781a793cea50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:39.522434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998aa3e-b1d4-4654-a5fc-781a793cea50 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:39.522713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998aa3e-b1d4-4654-a5fc-781a793cea50 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:39.523023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998aa3e-b1d4-4654-a5fc-781a793cea50 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:39.523588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998aa3e-b1d4-4654-a5fc-781a793cea50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:39.525733 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998aa3e-b1d4-4654-a5fc-781a793cea50 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:39.666589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1346/2688] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:30:39 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:39.671330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f4fce-084d-4967-9554-1bdf60be4172 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:39.682321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f4fce-084d-4967-9554-1bdf60be4172 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:39.687140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165088a7-e7e4-4ce5-a66c-d07896697496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:39.694013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165088a7-e7e4-4ce5-a66c-d07896697496 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:39.694013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165088a7-e7e4-4ce5-a66c-d07896697496 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:39.694013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165088a7-e7e4-4ce5-a66c-d07896697496 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:39.694013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165088a7-e7e4-4ce5-a66c-d07896697496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:39.694013 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165088a7-e7e4-4ce5-a66c-d07896697496 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:39.701832 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1343/2689] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:39.706261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998aa3e-b1d4-4654-a5fc-781a793cea50 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:39.716935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998aa3e-b1d4-4654-a5fc-781a793cea50 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:39.721526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62328823-a9de-4f24-8354-fa3768bcafb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:39.722271 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62328823-a9de-4f24-8354-fa3768bcafb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:39.722553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62328823-a9de-4f24-8354-fa3768bcafb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:39.722824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62328823-a9de-4f24-8354-fa3768bcafb5 admin admin] PATH_INFO: `/v3/projects/2bfac1a3a18040a3a74ef8699c2e0032/users/1210ab33fc8d40779b8f55f72eb5f56f/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:39.737866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62328823-a9de-4f24-8354-fa3768bcafb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bfac1a3a18040a3a74ef8699c2e0032, user_id=1210ab33fc8d40779b8f55f72eb5f56f, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:39.740520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62328823-a9de-4f24-8354-fa3768bcafb5 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:39.833581 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1347/2690] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:30:39 2022] PUT /identity/v3/projects/2bfac1a3a18040a3a74ef8699c2e0032/users/1210ab33fc8d40779b8f55f72eb5f56f/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:30:39.838798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165088a7-e7e4-4ce5-a66c-d07896697496 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:39.848516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165088a7-e7e4-4ce5-a66c-d07896697496 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:39.867568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:39.907110 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e556d-3674-4f4f-a242-a158e1137b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:39.907740 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e556d-3674-4f4f-a242-a158e1137b27 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:39.908010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e556d-3674-4f4f-a242-a158e1137b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:39.908274 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e556d-3674-4f4f-a242-a158e1137b27 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:39.910458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e556d-3674-4f4f-a242-a158e1137b27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:39.910580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e556d-3674-4f4f-a242-a158e1137b27 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:39.920710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1344/2691] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:39.925448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62328823-a9de-4f24-8354-fa3768bcafb5 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:39.931898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:39.937352 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62328823-a9de-4f24-8354-fa3768bcafb5 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:39.942214 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a5d2b-50ae-4d26-9f2f-56661d201c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:39.942858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a5d2b-50ae-4d26-9f2f-56661d201c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:39.943165 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a5d2b-50ae-4d26-9f2f-56661d201c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:39.943436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a5d2b-50ae-4d26-9f2f-56661d201c08 admin admin] PATH_INFO: `/v3/projects/2bfac1a3a18040a3a74ef8699c2e0032/users/1210ab33fc8d40779b8f55f72eb5f56f/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:39.945424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a5d2b-50ae-4d26-9f2f-56661d201c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bfac1a3a18040a3a74ef8699c2e0032, user_id=1210ab33fc8d40779b8f55f72eb5f56f, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:39.950046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a5d2b-50ae-4d26-9f2f-56661d201c08 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:39.960809 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1348/2692] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:30:39 2022] PUT /identity/v3/projects/2bfac1a3a18040a3a74ef8699c2e0032/users/1210ab33fc8d40779b8f55f72eb5f56f/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:39.972856 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f795fff-7e37-4832-8760-b92b7c076cca None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:39.973146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f795fff-7e37-4832-8760-b92b7c076cca None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:39.973427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f795fff-7e37-4832-8760-b92b7c076cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:40.002231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-4f795fff-7e37-4832-8760-b92b7c076cca None None] MFA Rules not processed for user `1210ab33fc8d40779b8f55f72eb5f56f`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:40.003488 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4f795fff-7e37-4832-8760-b92b7c076cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:40.074541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1345/2693] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:40.085584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a5d2b-50ae-4d26-9f2f-56661d201c08 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:40.087442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2b5a5d2b-50ae-4d26-9f2f-56661d201c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:40.148228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a5d2b-50ae-4d26-9f2f-56661d201c08 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:40.151983 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc7c17-5975-4d9a-8478-aaf056756f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:40.152719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc7c17-5975-4d9a-8478-aaf056756f1a service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:40.153024 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc7c17-5975-4d9a-8478-aaf056756f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:40.153276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc7c17-5975-4d9a-8478-aaf056756f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:40.162541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc7c17-5975-4d9a-8478-aaf056756f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:40.163962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc7c17-5975-4d9a-8478-aaf056756f1a service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:40.173540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1349/2694] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:30:43.104531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f795fff-7e37-4832-8760-b92b7c076cca None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:43.169260 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f795fff-7e37-4832-8760-b92b7c076cca None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:43.174989 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:43.175607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:43.175881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:43.176145 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] PATH_INFO: `/v3/users/83cff6ace98f467ca689284af33688b0` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:43.177256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=83cff6ace98f467ca689284af33688b0)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:43.179196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:43.349372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83cff6ace98f467ca689284af33688b0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:43.351045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:43.461081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83cff6ace98f467ca689284af33688b0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:43.468514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83cff6ace98f467ca689284af33688b0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:43.481335 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83cff6ace98f467ca689284af33688b0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:43.482078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83cff6ace98f467ca689284af33688b0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:43.488145 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1346/2695] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:30:43 2022] DELETE /identity/v3/users/83cff6ace98f467ca689284af33688b0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:43.493300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9a0a-2458-4645-ba7a-aabaa6afd8d7 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:43.493792 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9a0a-2458-4645-ba7a-aabaa6afd8d7 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:43.493888 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a9a0a-2458-4645-ba7a-aabaa6afd8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:43.512267 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-3c7a9a0a-2458-4645-ba7a-aabaa6afd8d7 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:43.513473 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3c7a9a0a-2458-4645-ba7a-aabaa6afd8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:43.577309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1350/2696] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:43.587122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:43.588867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:43.650423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c5340-318c-4fb6-b900-a4e1c3efa0cc admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:43.655113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7de54-9ee8-4efc-a375-7206c8555a97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:43.655703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7de54-9ee8-4efc-a375-7206c8555a97 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:43.655980 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7de54-9ee8-4efc-a375-7206c8555a97 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:43.656248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7de54-9ee8-4efc-a375-7206c8555a97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:43.665751 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7de54-9ee8-4efc-a375-7206c8555a97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:43.667217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7de54-9ee8-4efc-a375-7206c8555a97 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:43.676581 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1347/2697] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:30:44.091316 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9a0a-2458-4645-ba7a-aabaa6afd8d7 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:44.091316 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3c7a9a0a-2458-4645-ba7a-aabaa6afd8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:44.100892 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a9a0a-2458-4645-ba7a-aabaa6afd8d7 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:44.106117 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:44.106738 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:44.107011 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:44.107277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] PATH_INFO: `/v3/projects/838cfd8867b3413a9b26ff40af8cc818` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:44.108450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=838cfd8867b3413a9b26ff40af8cc818)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:44.110331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:44.222510 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '838cfd8867b3413a9b26ff40af8cc818', 'request_id': 'req-62470d3f-a68b-4ff2-b759-565951f695cc'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:44.293881 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '838cfd8867b3413a9b26ff40af8cc818', 'request_id': 'req-62470d3f-a68b-4ff2-b759-565951f695cc'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:44.300645 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '838cfd8867b3413a9b26ff40af8cc818', 'request_id': 'req-62470d3f-a68b-4ff2-b759-565951f695cc'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:44.311034 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] The token cache is being invalidate because project 838cfd8867b3413a9b26ff40af8cc818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:30:44.311314 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:44.311932 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:44.313390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1351/2698] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:30:44 2022] DELETE /identity/v3/projects/838cfd8867b3413a9b26ff40af8cc818 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:44.317286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7de54-9ee8-4efc-a375-7206c8555a97 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:44.327250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7de54-9ee8-4efc-a375-7206c8555a97 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:44.381496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:44.382137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:44.382414 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:44.382681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] PATH_INFO: `/v3/users/1210ab33fc8d40779b8f55f72eb5f56f` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:44.384321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1210ab33fc8d40779b8f55f72eb5f56f)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:44.386291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:44.622695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1210ab33fc8d40779b8f55f72eb5f56f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:44.710815 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1210ab33fc8d40779b8f55f72eb5f56f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:44.718368 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1210ab33fc8d40779b8f55f72eb5f56f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:44.723540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:44.730369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1210ab33fc8d40779b8f55f72eb5f56f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:44.731089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-321b8c34-7c05-45fa-9b20-a3591e2ceb31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1210ab33fc8d40779b8f55f72eb5f56f'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:44.737232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1348/2699] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:30:44 2022] DELETE /identity/v3/users/1210ab33fc8d40779b8f55f72eb5f56f => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:45.156889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:45.158458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:45.172130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62470d3f-a68b-4ff2-b759-565951f695cc admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:45.189718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:45.231374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:45.232026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:45.232305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:45.232631 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] PATH_INFO: `/v3/projects/2bfac1a3a18040a3a74ef8699c2e0032` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:45.233753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bfac1a3a18040a3a74ef8699c2e0032)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:45.235708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:45.381674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bfac1a3a18040a3a74ef8699c2e0032', 'request_id': 'req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:45.471154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bfac1a3a18040a3a74ef8699c2e0032', 'request_id': 'req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:45.478819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bfac1a3a18040a3a74ef8699c2e0032', 'request_id': 'req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:45.488998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] The token cache is being invalidate because project 2bfac1a3a18040a3a74ef8699c2e0032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:30:45.489277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:45.489913 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-18b1b3a6-46d2-46f5-96cb-e07b7a922df9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:30:45.491408 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1352/2700] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:30:45 2022] DELETE /identity/v3/projects/2bfac1a3a18040a3a74ef8699c2e0032 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:45.496524 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c979c3-8f71-40a5-914e-bc0186bc92a3 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:45.497006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c979c3-8f71-40a5-914e-bc0186bc92a3 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:45.497124 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c979c3-8f71-40a5-914e-bc0186bc92a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:45.516995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a7c979c3-8f71-40a5-914e-bc0186bc92a3 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:45.518263 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a7c979c3-8f71-40a5-914e-bc0186bc92a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:45.579033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1349/2701] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:45.588683 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ff8be-700c-4643-8a83-14d895acaccd None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:45.588964 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ff8be-700c-4643-8a83-14d895acaccd None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:45.589238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16ff8be-700c-4643-8a83-14d895acaccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:45.609484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b16ff8be-700c-4643-8a83-14d895acaccd None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:45.610737 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b16ff8be-700c-4643-8a83-14d895acaccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:45.619140 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1353/2702] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:45.623231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c979c3-8f71-40a5-914e-bc0186bc92a3 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:45.632954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c979c3-8f71-40a5-914e-bc0186bc92a3 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:45.637340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7446e3-8fb3-43b7-9a39-28a458d02c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:45.637926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7446e3-8fb3-43b7-9a39-28a458d02c85 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:45.638230 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7446e3-8fb3-43b7-9a39-28a458d02c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:45.638495 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7446e3-8fb3-43b7-9a39-28a458d02c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:45.639122 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7446e3-8fb3-43b7-9a39-28a458d02c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:30:45.639415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7446e3-8fb3-43b7-9a39-28a458d02c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:45.640970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7446e3-8fb3-43b7-9a39-28a458d02c85 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:45.652896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1350/2703] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:30:45.657208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ff8be-700c-4643-8a83-14d895acaccd None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:45.667130 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16ff8be-700c-4643-8a83-14d895acaccd None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:45.671867 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbdc08-548c-4968-836e-fd695cd4ee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:45.672453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbdc08-548c-4968-836e-fd695cd4ee3c admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:45.672729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbdc08-548c-4968-836e-fd695cd4ee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:45.672995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbdc08-548c-4968-836e-fd695cd4ee3c admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:45.673604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbdc08-548c-4968-836e-fd695cd4ee3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:45.675513 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bbdc08-548c-4968-836e-fd695cd4ee3c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:45.794227 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1354/2704] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:30:45 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:45.798774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7446e3-8fb3-43b7-9a39-28a458d02c85 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:45.809024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7446e3-8fb3-43b7-9a39-28a458d02c85 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:45.863408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd2392-9459-4d22-b889-8f5ece23818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:45.863997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd2392-9459-4d22-b889-8f5ece23818d admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:45.864270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd2392-9459-4d22-b889-8f5ece23818d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:45.864536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd2392-9459-4d22-b889-8f5ece23818d admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:45.865151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd2392-9459-4d22-b889-8f5ece23818d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:45.867081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd2392-9459-4d22-b889-8f5ece23818d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:45.964891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1351/2705] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:30:45 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:30:45.968991 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbdc08-548c-4968-836e-fd695cd4ee3c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:45.979570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbdc08-548c-4968-836e-fd695cd4ee3c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:45.984169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa55f6-71bc-41e8-93d5-a905f36cb805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:45.984753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa55f6-71bc-41e8-93d5-a905f36cb805 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:45.985029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa55f6-71bc-41e8-93d5-a905f36cb805 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:45.985294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa55f6-71bc-41e8-93d5-a905f36cb805 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:45.985928 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa55f6-71bc-41e8-93d5-a905f36cb805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:45.987509 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa55f6-71bc-41e8-93d5-a905f36cb805 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:45.997707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1355/2706] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:46.001766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd2392-9459-4d22-b889-8f5ece23818d admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:46.012246 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd2392-9459-4d22-b889-8f5ece23818d admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:46.016796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802511db-2869-4945-83a7-1174d0a2d6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:46.017377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802511db-2869-4945-83a7-1174d0a2d6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:46.017652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802511db-2869-4945-83a7-1174d0a2d6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:46.017920 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802511db-2869-4945-83a7-1174d0a2d6c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:46.018584 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802511db-2869-4945-83a7-1174d0a2d6c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:46.020136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802511db-2869-4945-83a7-1174d0a2d6c9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:46.031106 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1352/2707] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:46.035263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa55f6-71bc-41e8-93d5-a905f36cb805 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:46.045171 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa55f6-71bc-41e8-93d5-a905f36cb805 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:46.049638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eadcc5-1512-46a3-902c-b4ae1d8b63e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:46.050297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eadcc5-1512-46a3-902c-b4ae1d8b63e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:46.050570 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eadcc5-1512-46a3-902c-b4ae1d8b63e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:46.050837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eadcc5-1512-46a3-902c-b4ae1d8b63e1 admin admin] PATH_INFO: `/v3/projects/c56701bed2ba47febbbf39d61e6434f0/users/dbd91b4e08b74508b50d92760ea964c2/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:46.065824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eadcc5-1512-46a3-902c-b4ae1d8b63e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56701bed2ba47febbbf39d61e6434f0, user_id=dbd91b4e08b74508b50d92760ea964c2, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:46.068459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eadcc5-1512-46a3-902c-b4ae1d8b63e1 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:46.141744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1356/2708] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:30:46 2022] PUT /identity/v3/projects/c56701bed2ba47febbbf39d61e6434f0/users/dbd91b4e08b74508b50d92760ea964c2/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:46.152108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4aafb4-1c10-47c5-a00a-a232d2e05d56 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:46.152391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4aafb4-1c10-47c5-a00a-a232d2e05d56 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:46.152660 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4aafb4-1c10-47c5-a00a-a232d2e05d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:46.181222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-de4aafb4-1c10-47c5-a00a-a232d2e05d56 None None] MFA Rules not processed for user `dbd91b4e08b74508b50d92760ea964c2`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:46.182365 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-de4aafb4-1c10-47c5-a00a-a232d2e05d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:46.221670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:46.244511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1353/2709] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:46.249202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eadcc5-1512-46a3-902c-b4ae1d8b63e1 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:46.293367 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:46.312080 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eadcc5-1512-46a3-902c-b4ae1d8b63e1 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:46.316778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c12a1f-752a-4d65-bf66-46ffb7735020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:46.317370 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c12a1f-752a-4d65-bf66-46ffb7735020 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:46.317650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c12a1f-752a-4d65-bf66-46ffb7735020 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:46.317918 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c12a1f-752a-4d65-bf66-46ffb7735020 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:46.318576 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c12a1f-752a-4d65-bf66-46ffb7735020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:46.320453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c12a1f-752a-4d65-bf66-46ffb7735020 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:46.428755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1357/2710] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:30:46 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:30:46.434028 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4aafb4-1c10-47c5-a00a-a232d2e05d56 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:46.444951 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4aafb4-1c10-47c5-a00a-a232d2e05d56 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:46.500911 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376ad26-76c4-4df6-ab9b-51cbc4494af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:46.501598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376ad26-76c4-4df6-ab9b-51cbc4494af9 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:46.501772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376ad26-76c4-4df6-ab9b-51cbc4494af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:46.502076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376ad26-76c4-4df6-ab9b-51cbc4494af9 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:46.502694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376ad26-76c4-4df6-ab9b-51cbc4494af9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:46.504676 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376ad26-76c4-4df6-ab9b-51cbc4494af9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:46.593421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1354/2711] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:30:46 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:30:46.597397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c12a1f-752a-4d65-bf66-46ffb7735020 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:46.608295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c12a1f-752a-4d65-bf66-46ffb7735020 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:46.613651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb368ab-5d9c-4170-87dd-e9f9b9cc0c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:46.615560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb368ab-5d9c-4170-87dd-e9f9b9cc0c15 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:46.615560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb368ab-5d9c-4170-87dd-e9f9b9cc0c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:46.615560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb368ab-5d9c-4170-87dd-e9f9b9cc0c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:46.615560 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb368ab-5d9c-4170-87dd-e9f9b9cc0c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:46.617005 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb368ab-5d9c-4170-87dd-e9f9b9cc0c15 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:46.627956 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1358/2712] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:46.633342 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376ad26-76c4-4df6-ab9b-51cbc4494af9 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:46.644232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376ad26-76c4-4df6-ab9b-51cbc4494af9 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:46.650020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea43666-c4b5-4312-9b44-0d4c391469e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:46.650020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea43666-c4b5-4312-9b44-0d4c391469e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:46.650257 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea43666-c4b5-4312-9b44-0d4c391469e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:46.650463 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea43666-c4b5-4312-9b44-0d4c391469e7 admin admin] PATH_INFO: `/v3/projects/85cb7df74d434905a1dadabbd634bc05/users/aa6f67e7d1834b42a97a335b905904e1/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:46.666250 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea43666-c4b5-4312-9b44-0d4c391469e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85cb7df74d434905a1dadabbd634bc05, user_id=aa6f67e7d1834b42a97a335b905904e1, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:46.668588 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea43666-c4b5-4312-9b44-0d4c391469e7 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:46.757371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1355/2713] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:30:46 2022] PUT /identity/v3/projects/85cb7df74d434905a1dadabbd634bc05/users/aa6f67e7d1834b42a97a335b905904e1/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:46.761403 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb368ab-5d9c-4170-87dd-e9f9b9cc0c15 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:46.771468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb368ab-5d9c-4170-87dd-e9f9b9cc0c15 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:46.829958 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0985107-03c5-4a43-bd12-0163507bb782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:46.830573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0985107-03c5-4a43-bd12-0163507bb782 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:46.830852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0985107-03c5-4a43-bd12-0163507bb782 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:46.831129 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0985107-03c5-4a43-bd12-0163507bb782 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:46.831774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0985107-03c5-4a43-bd12-0163507bb782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:46.833351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0985107-03c5-4a43-bd12-0163507bb782 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:46.843798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1359/2714] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:30:46.848037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea43666-c4b5-4312-9b44-0d4c391469e7 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:46.858927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea43666-c4b5-4312-9b44-0d4c391469e7 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:46.863483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af0dda-f051-4ba9-962b-967699fabba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:46.864109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af0dda-f051-4ba9-962b-967699fabba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:46.864384 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af0dda-f051-4ba9-962b-967699fabba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:46.864652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af0dda-f051-4ba9-962b-967699fabba7 admin admin] PATH_INFO: `/v3/projects/85cb7df74d434905a1dadabbd634bc05/users/aa6f67e7d1834b42a97a335b905904e1/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:46.866635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af0dda-f051-4ba9-962b-967699fabba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85cb7df74d434905a1dadabbd634bc05, user_id=aa6f67e7d1834b42a97a335b905904e1, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:46.869228 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af0dda-f051-4ba9-962b-967699fabba7 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:46.879216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1356/2715] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:30:46 2022] PUT /identity/v3/projects/85cb7df74d434905a1dadabbd634bc05/users/aa6f67e7d1834b42a97a335b905904e1/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:30:46.890191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52444cf-143e-4798-807d-0b98ac95ccca None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:46.890476 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52444cf-143e-4798-807d-0b98ac95ccca None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:46.890745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52444cf-143e-4798-807d-0b98ac95ccca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:46.918897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b52444cf-143e-4798-807d-0b98ac95ccca None None] MFA Rules not processed for user `aa6f67e7d1834b42a97a335b905904e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:30:46.920099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b52444cf-143e-4798-807d-0b98ac95ccca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:46.991430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1360/2716] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:30:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:30:47.002852 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af0dda-f051-4ba9-962b-967699fabba7 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:47.004917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e6af0dda-f051-4ba9-962b-967699fabba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:30:47.065098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af0dda-f051-4ba9-962b-967699fabba7 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:47.068713 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f8a0b-c77f-4b3a-969b-e0e1c0f3a9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:47.069306 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f8a0b-c77f-4b3a-969b-e0e1c0f3a9ed service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:47.069580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f8a0b-c77f-4b3a-969b-e0e1c0f3a9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:47.069847 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f8a0b-c77f-4b3a-969b-e0e1c0f3a9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:47.115468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:30:47.124725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f8a0b-c77f-4b3a-969b-e0e1c0f3a9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:47.126199 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f8a0b-c77f-4b3a-969b-e0e1c0f3a9ed service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:47.134853 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1357/2717] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:30:54.516903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52444cf-143e-4798-807d-0b98ac95ccca None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:30:54.527375 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52444cf-143e-4798-807d-0b98ac95ccca None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:30:54.530957 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9acf41-b365-401a-b566-11f8a55101f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:30:54.531555 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9acf41-b365-401a-b566-11f8a55101f3 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:30:54.531859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9acf41-b365-401a-b566-11f8a55101f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:30:54.532096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9acf41-b365-401a-b566-11f8a55101f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:30:54.543440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9acf41-b365-401a-b566-11f8a55101f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:30:54.544859 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9acf41-b365-401a-b566-11f8a55101f3 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:30:54.554786 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1361/2718] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:30:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:30:57.175182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:04.550332 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=88,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:30.125229 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f8a0b-c77f-4b3a-969b-e0e1c0f3a9ed service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:30.226397 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f8a0b-c77f-4b3a-969b-e0e1c0f3a9ed service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:30.231955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:30.232569 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:30.232851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:30.233123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] PATH_INFO: `/v3/users/dbd91b4e08b74508b50d92760ea964c2` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:30.234266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbd91b4e08b74508b50d92760ea964c2)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:30.236143 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:30.405100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd91b4e08b74508b50d92760ea964c2'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:30.522419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd91b4e08b74508b50d92760ea964c2'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:30.530684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd91b4e08b74508b50d92760ea964c2'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:30.544326 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd91b4e08b74508b50d92760ea964c2'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:30.545119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd91b4e08b74508b50d92760ea964c2'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:30.551685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1358/2719] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:31:30 2022] DELETE /identity/v3/users/dbd91b4e08b74508b50d92760ea964c2 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:30.556385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc079f-ac16-4a0e-ae97-ea6bba747674 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:30.556883 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc079f-ac16-4a0e-ae97-ea6bba747674 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:30.556978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc079f-ac16-4a0e-ae97-ea6bba747674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:30.579238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-fadc079f-ac16-4a0e-ae97-ea6bba747674 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:30.580430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fadc079f-ac16-4a0e-ae97-ea6bba747674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:30.638306 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1362/2720] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:30.648039 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:30.649418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:30.710766 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb925c08-af78-49f0-bdd7-470adc547669 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:30.714255 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afef288-d38f-4c5e-b549-5bc52f894b37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:30.714835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afef288-d38f-4c5e-b549-5bc52f894b37 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:30.715618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afef288-d38f-4c5e-b549-5bc52f894b37 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:30.715880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afef288-d38f-4c5e-b549-5bc52f894b37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:30.724458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afef288-d38f-4c5e-b549-5bc52f894b37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:30.725851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afef288-d38f-4c5e-b549-5bc52f894b37 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:30.729639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:30.736164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1359/2721] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:31:31.141938 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc079f-ac16-4a0e-ae97-ea6bba747674 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:31.143554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-fadc079f-ac16-4a0e-ae97-ea6bba747674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:31.156693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc079f-ac16-4a0e-ae97-ea6bba747674 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:31.161441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:31.162114 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:31.162436 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:31.162610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] PATH_INFO: `/v3/projects/c56701bed2ba47febbbf39d61e6434f0` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:31.163726 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56701bed2ba47febbbf39d61e6434f0)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:31.165602 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:31.275862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56701bed2ba47febbbf39d61e6434f0', 'request_id': 'req-59fbf40f-faa3-4346-a702-23fa851e52bb'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:31.354995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56701bed2ba47febbbf39d61e6434f0', 'request_id': 'req-59fbf40f-faa3-4346-a702-23fa851e52bb'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:31.361943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56701bed2ba47febbbf39d61e6434f0', 'request_id': 'req-59fbf40f-faa3-4346-a702-23fa851e52bb'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:31.373731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] The token cache is being invalidate because project c56701bed2ba47febbbf39d61e6434f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:31:31.375687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:31.375687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:31.376169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1363/2722] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:31:31 2022] DELETE /identity/v3/projects/c56701bed2ba47febbbf39d61e6434f0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:31.380375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afef288-d38f-4c5e-b549-5bc52f894b37 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:31.391222 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afef288-d38f-4c5e-b549-5bc52f894b37 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:31.447592 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:31.448204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:31.448476 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:31.448740 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] PATH_INFO: `/v3/users/aa6f67e7d1834b42a97a335b905904e1` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:31.450639 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa6f67e7d1834b42a97a335b905904e1)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:31.452575 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:31.685492 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa6f67e7d1834b42a97a335b905904e1'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:31.760265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa6f67e7d1834b42a97a335b905904e1'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:31.768577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa6f67e7d1834b42a97a335b905904e1'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:31.781135 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa6f67e7d1834b42a97a335b905904e1'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:31.781829 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-8677ab6f-58b5-4513-ae3f-b1aa288da0f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa6f67e7d1834b42a97a335b905904e1'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:31.787722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1360/2723] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:31:31 2022] DELETE /identity/v3/users/aa6f67e7d1834b42a97a335b905904e1 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:32.419211 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:32.420742 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:32.435797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fbf40f-faa3-4346-a702-23fa851e52bb admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:32.496132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:32.501084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:32.501084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:32.501084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] PATH_INFO: `/v3/projects/85cb7df74d434905a1dadabbd634bc05` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:32.501084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=85cb7df74d434905a1dadabbd634bc05)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:32.501084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:32.584413 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:32.601995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85cb7df74d434905a1dadabbd634bc05', 'request_id': 'req-e3b2ab19-30bc-4998-85c9-704da4900d5a'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:32.668380 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85cb7df74d434905a1dadabbd634bc05', 'request_id': 'req-e3b2ab19-30bc-4998-85c9-704da4900d5a'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:32.674874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85cb7df74d434905a1dadabbd634bc05', 'request_id': 'req-e3b2ab19-30bc-4998-85c9-704da4900d5a'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:32.689655 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] The token cache is being invalidate because project 85cb7df74d434905a1dadabbd634bc05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:31:32.689946 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:32.690629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e3b2ab19-30bc-4998-85c9-704da4900d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:32.692123 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1364/2724] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:31:32 2022] DELETE /identity/v3/projects/85cb7df74d434905a1dadabbd634bc05 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:31:32.696927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257224e-9846-4031-a13f-f5a2287a3ebf None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:32.707069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257224e-9846-4031-a13f-f5a2287a3ebf None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:32.707069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257224e-9846-4031-a13f-f5a2287a3ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:32.718148 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-a257224e-9846-4031-a13f-f5a2287a3ebf None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:32.719336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-a257224e-9846-4031-a13f-f5a2287a3ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:32.778861 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1361/2725] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:32.786763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1dd1-3c7b-488d-b4b1-becc63e8b3a8 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:32.787042 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1dd1-3c7b-488d-b4b1-becc63e8b3a8 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:32.787307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb1dd1-3c7b-488d-b4b1-becc63e8b3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:32.806624 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f7eb1dd1-3c7b-488d-b4b1-becc63e8b3a8 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:32.807803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f7eb1dd1-3c7b-488d-b4b1-becc63e8b3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:32.816038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1365/2726] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:32.820424 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257224e-9846-4031-a13f-f5a2287a3ebf None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:32.831025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257224e-9846-4031-a13f-f5a2287a3ebf None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:32.835548 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83199f-3cca-4a28-bb56-fd671ea756b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:32.836129 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83199f-3cca-4a28-bb56-fd671ea756b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:32.836405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83199f-3cca-4a28-bb56-fd671ea756b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:32.836673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83199f-3cca-4a28-bb56-fd671ea756b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:32.837336 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83199f-3cca-4a28-bb56-fd671ea756b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:31:32.837633 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83199f-3cca-4a28-bb56-fd671ea756b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:32.839438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83199f-3cca-4a28-bb56-fd671ea756b4 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:32.850880 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1362/2727] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:31:32.855392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1dd1-3c7b-488d-b4b1-becc63e8b3a8 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:32.864622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb1dd1-3c7b-488d-b4b1-becc63e8b3a8 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:32.869377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad41ae1-f84b-4468-84f7-a2ace33c095f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:32.869980 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad41ae1-f84b-4468-84f7-a2ace33c095f admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:32.870276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad41ae1-f84b-4468-84f7-a2ace33c095f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:32.870542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad41ae1-f84b-4468-84f7-a2ace33c095f admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:32.871181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad41ae1-f84b-4468-84f7-a2ace33c095f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:32.873612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad41ae1-f84b-4468-84f7-a2ace33c095f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:32.961574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1366/2728] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:31:32 2022] POST /identity/v3/projects => generated 393 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:31:32.965848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83199f-3cca-4a28-bb56-fd671ea756b4 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:32.972227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:32.976069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83199f-3cca-4a28-bb56-fd671ea756b4 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:33.032378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf09f0c-23a6-4eb1-b10e-e17e7fc70a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:33.032887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf09f0c-23a6-4eb1-b10e-e17e7fc70a4b admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:33.033160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf09f0c-23a6-4eb1-b10e-e17e7fc70a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:33.033425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf09f0c-23a6-4eb1-b10e-e17e7fc70a4b admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:33.034069 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf09f0c-23a6-4eb1-b10e-e17e7fc70a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:33.035946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf09f0c-23a6-4eb1-b10e-e17e7fc70a4b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:33.123955 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1363/2729] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:31:32 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:31:33.127896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad41ae1-f84b-4468-84f7-a2ace33c095f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:33.139209 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad41ae1-f84b-4468-84f7-a2ace33c095f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:33.143861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752260e4-3653-4fd8-a9fb-176c20d54850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:33.148894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752260e4-3653-4fd8-a9fb-176c20d54850 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:33.148894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752260e4-3653-4fd8-a9fb-176c20d54850 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:33.148894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752260e4-3653-4fd8-a9fb-176c20d54850 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:33.148894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752260e4-3653-4fd8-a9fb-176c20d54850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:33.148894 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752260e4-3653-4fd8-a9fb-176c20d54850 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:33.158489 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1367/2730] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:33.162574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf09f0c-23a6-4eb1-b10e-e17e7fc70a4b admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:33.172625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf09f0c-23a6-4eb1-b10e-e17e7fc70a4b admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:33.177145 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6496b41-b36e-4304-81f2-41114a26e747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:33.177727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6496b41-b36e-4304-81f2-41114a26e747 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:33.178029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6496b41-b36e-4304-81f2-41114a26e747 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:33.178298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6496b41-b36e-4304-81f2-41114a26e747 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:33.179186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6496b41-b36e-4304-81f2-41114a26e747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:33.180759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6496b41-b36e-4304-81f2-41114a26e747 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:33.191917 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1364/2731] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:33.196243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752260e4-3653-4fd8-a9fb-176c20d54850 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:33.207008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752260e4-3653-4fd8-a9fb-176c20d54850 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:33.213585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f418c-97c4-485f-941c-09d20f2548d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:33.214616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f418c-97c4-485f-941c-09d20f2548d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:33.214893 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f418c-97c4-485f-941c-09d20f2548d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:33.215163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f418c-97c4-485f-941c-09d20f2548d8 admin admin] PATH_INFO: `/v3/projects/ddec3ec555a24bb59559b553d060d802/users/989c1ced75ca4055877e3ff71b0e7bfa/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:33.229872 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f418c-97c4-485f-941c-09d20f2548d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddec3ec555a24bb59559b553d060d802, user_id=989c1ced75ca4055877e3ff71b0e7bfa, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:33.233060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81f418c-97c4-485f-941c-09d20f2548d8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:33.306357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1368/2732] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:31:33 2022] PUT /identity/v3/projects/ddec3ec555a24bb59559b553d060d802/users/989c1ced75ca4055877e3ff71b0e7bfa/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:33.318174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f1501-a8f7-49a5-9c9b-2857e6bbf311 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:33.318419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f1501-a8f7-49a5-9c9b-2857e6bbf311 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:33.318687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f1501-a8f7-49a5-9c9b-2857e6bbf311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:33.349757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-441f1501-a8f7-49a5-9c9b-2857e6bbf311 None None] MFA Rules not processed for user `989c1ced75ca4055877e3ff71b0e7bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:33.350929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-441f1501-a8f7-49a5-9c9b-2857e6bbf311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:33.416800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1365/2733] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:33.421935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f418c-97c4-485f-941c-09d20f2548d8 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:33.488638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f418c-97c4-485f-941c-09d20f2548d8 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:33.493685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c2fd4-6be6-4d5d-b416-35d70292165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:33.494307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c2fd4-6be6-4d5d-b416-35d70292165a admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:33.494573 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c2fd4-6be6-4d5d-b416-35d70292165a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:33.494835 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c2fd4-6be6-4d5d-b416-35d70292165a admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:33.495477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c2fd4-6be6-4d5d-b416-35d70292165a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:33.497969 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c2fd4-6be6-4d5d-b416-35d70292165a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:33.578954 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1369/2734] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:31:33 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:31:33.580780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f1501-a8f7-49a5-9c9b-2857e6bbf311 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:33.593216 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f1501-a8f7-49a5-9c9b-2857e6bbf311 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:33.650922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80a873-e671-4a41-8e9f-d73b34d16bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:33.651511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80a873-e671-4a41-8e9f-d73b34d16bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:33.651787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80a873-e671-4a41-8e9f-d73b34d16bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:33.652055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80a873-e671-4a41-8e9f-d73b34d16bb5 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:33.653188 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80a873-e671-4a41-8e9f-d73b34d16bb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:33.655117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80a873-e671-4a41-8e9f-d73b34d16bb5 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:33.771595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1366/2735] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:31:33 2022] POST /identity/v3/users => generated 303 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:31:33.775572 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c2fd4-6be6-4d5d-b416-35d70292165a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:33.786753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c2fd4-6be6-4d5d-b416-35d70292165a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:33.791501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cf171-7fee-47b2-af0d-8dd3e28b7b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:33.792093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cf171-7fee-47b2-af0d-8dd3e28b7b4b admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:33.792389 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cf171-7fee-47b2-af0d-8dd3e28b7b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:33.792663 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cf171-7fee-47b2-af0d-8dd3e28b7b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:33.793318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cf171-7fee-47b2-af0d-8dd3e28b7b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:33.795574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cf171-7fee-47b2-af0d-8dd3e28b7b4b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:33.807108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1370/2736] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:33.811426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80a873-e671-4a41-8e9f-d73b34d16bb5 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:33.816640 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:33.822050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80a873-e671-4a41-8e9f-d73b34d16bb5 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:33.837681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0894f-3a38-46d1-9314-e52f4d01e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:33.837681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0894f-3a38-46d1-9314-e52f4d01e7ee admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:33.837681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0894f-3a38-46d1-9314-e52f4d01e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:33.837681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0894f-3a38-46d1-9314-e52f4d01e7ee admin admin] PATH_INFO: `/v3/projects/204bdde2dde847bda042c23f9b3f3859/users/1417ec2fa06a433885bb8770c48a9b92/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:33.844150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0894f-3a38-46d1-9314-e52f4d01e7ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=204bdde2dde847bda042c23f9b3f3859, user_id=1417ec2fa06a433885bb8770c48a9b92, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:33.846806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0894f-3a38-46d1-9314-e52f4d01e7ee admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:33.934781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1367/2737] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:31:33 2022] PUT /identity/v3/projects/204bdde2dde847bda042c23f9b3f3859/users/1417ec2fa06a433885bb8770c48a9b92/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:33.940387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cf171-7fee-47b2-af0d-8dd3e28b7b4b admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:33.951173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cf171-7fee-47b2-af0d-8dd3e28b7b4b admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:33.972067 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:34.007341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184815f9-6335-4164-93b8-eb95ddb77076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:34.007918 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184815f9-6335-4164-93b8-eb95ddb77076 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:34.008203 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184815f9-6335-4164-93b8-eb95ddb77076 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:34.008463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184815f9-6335-4164-93b8-eb95ddb77076 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:34.009091 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184815f9-6335-4164-93b8-eb95ddb77076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:34.010693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184815f9-6335-4164-93b8-eb95ddb77076 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:34.020882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1371/2738] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:34.025241 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0894f-3a38-46d1-9314-e52f4d01e7ee admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:34.036146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0894f-3a38-46d1-9314-e52f4d01e7ee admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:34.041070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b2ee8-59cb-4437-b5a9-c6afe767f11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:34.041782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b2ee8-59cb-4437-b5a9-c6afe767f11e admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:34.042094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b2ee8-59cb-4437-b5a9-c6afe767f11e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:34.042363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b2ee8-59cb-4437-b5a9-c6afe767f11e admin admin] PATH_INFO: `/v3/projects/204bdde2dde847bda042c23f9b3f3859/users/1417ec2fa06a433885bb8770c48a9b92/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:34.044408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b2ee8-59cb-4437-b5a9-c6afe767f11e admin admin] RBAC: Authorizing `identity:create_grant(project_id=204bdde2dde847bda042c23f9b3f3859, user_id=1417ec2fa06a433885bb8770c48a9b92, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:34.047067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b2ee8-59cb-4437-b5a9-c6afe767f11e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:34.058505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1368/2739] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:31:34 2022] PUT /identity/v3/projects/204bdde2dde847bda042c23f9b3f3859/users/1417ec2fa06a433885bb8770c48a9b92/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:34.068507 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57d165-2c8c-4110-83ab-92986ccc92f8 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:34.068790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57d165-2c8c-4110-83ab-92986ccc92f8 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:34.069099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57d165-2c8c-4110-83ab-92986ccc92f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:34.097415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-aa57d165-2c8c-4110-83ab-92986ccc92f8 None None] MFA Rules not processed for user `1417ec2fa06a433885bb8770c48a9b92`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:34.098577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-aa57d165-2c8c-4110-83ab-92986ccc92f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:34.167414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1372/2740] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:34 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:34.178026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b2ee8-59cb-4437-b5a9-c6afe767f11e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:34.180037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bd0b2ee8-59cb-4437-b5a9-c6afe767f11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:34.239835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b2ee8-59cb-4437-b5a9-c6afe767f11e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:34.243686 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62abefb-f5cb-48b6-a6a1-b6273e9e7f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:34.244276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62abefb-f5cb-48b6-a6a1-b6273e9e7f3e service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:34.244547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62abefb-f5cb-48b6-a6a1-b6273e9e7f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:34.244811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62abefb-f5cb-48b6-a6a1-b6273e9e7f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:34.254507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62abefb-f5cb-48b6-a6a1-b6273e9e7f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:34.255906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62abefb-f5cb-48b6-a6a1-b6273e9e7f3e service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:34.266543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1369/2741] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:31:34 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:31:34.318043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57d165-2c8c-4110-83ab-92986ccc92f8 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:34.319706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-aa57d165-2c8c-4110-83ab-92986ccc92f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:34.331587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57d165-2c8c-4110-83ab-92986ccc92f8 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:34.335257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9107d397-c29e-4da8-8607-fe2b491e9807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:34.335842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107d397-c29e-4da8-8607-fe2b491e9807 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:34.336115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107d397-c29e-4da8-8607-fe2b491e9807 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:34.336383 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9107d397-c29e-4da8-8607-fe2b491e9807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:34.396333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107d397-c29e-4da8-8607-fe2b491e9807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:34.397617 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9107d397-c29e-4da8-8607-fe2b491e9807 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:34.408777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1373/2742] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:31:34 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:31:39.322541 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62abefb-f5cb-48b6-a6a1-b6273e9e7f3e service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:39.387764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62abefb-f5cb-48b6-a6a1-b6273e9e7f3e service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:39.392400 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:39.401346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:39.401346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:39.401346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] PATH_INFO: `/v3/users/989c1ced75ca4055877e3ff71b0e7bfa` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:39.401346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=989c1ced75ca4055877e3ff71b0e7bfa)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:39.401346 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:39.601159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '989c1ced75ca4055877e3ff71b0e7bfa'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:39.694458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '989c1ced75ca4055877e3ff71b0e7bfa'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:39.701835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '989c1ced75ca4055877e3ff71b0e7bfa'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:39.706970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:39.713711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '989c1ced75ca4055877e3ff71b0e7bfa'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:39.714376 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '989c1ced75ca4055877e3ff71b0e7bfa'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:39.721153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1370/2743] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:31:39 2022] DELETE /identity/v3/users/989c1ced75ca4055877e3ff71b0e7bfa => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:39.725712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ac46-793b-4c7b-ac51-e7e60ba26aa0 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:39.726275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ac46-793b-4c7b-ac51-e7e60ba26aa0 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:39.726275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ac46-793b-4c7b-ac51-e7e60ba26aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:39.745373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ab26ac46-793b-4c7b-ac51-e7e60ba26aa0 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:39.746599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ab26ac46-793b-4c7b-ac51-e7e60ba26aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:39.790516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:39.810308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1374/2744] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:39.820287 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:39.821929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:39.879233 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71f3e9-d64b-4c64-a84b-4cd18adea7d3 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:39.882811 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8e942-5526-40b8-976c-f5f1ab8fd6a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:39.883430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8e942-5526-40b8-976c-f5f1ab8fd6a7 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:39.883861 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8e942-5526-40b8-976c-f5f1ab8fd6a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:39.884081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8e942-5526-40b8-976c-f5f1ab8fd6a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:39.893477 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8e942-5526-40b8-976c-f5f1ab8fd6a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:39.894966 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8e942-5526-40b8-976c-f5f1ab8fd6a7 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:39.905625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1371/2745] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:31:40.299901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26ac46-793b-4c7b-ac51-e7e60ba26aa0 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:40.301473 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ab26ac46-793b-4c7b-ac51-e7e60ba26aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:40.314495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26ac46-793b-4c7b-ac51-e7e60ba26aa0 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:40.319337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:40.319974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:40.320249 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:40.320517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] PATH_INFO: `/v3/projects/ddec3ec555a24bb59559b553d060d802` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:40.321674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddec3ec555a24bb59559b553d060d802)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:40.323658 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:40.435855 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddec3ec555a24bb59559b553d060d802', 'request_id': 'req-b8b7711a-68eb-443d-bacb-089fca3860e6'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:40.509256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddec3ec555a24bb59559b553d060d802', 'request_id': 'req-b8b7711a-68eb-443d-bacb-089fca3860e6'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:40.517698 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddec3ec555a24bb59559b553d060d802', 'request_id': 'req-b8b7711a-68eb-443d-bacb-089fca3860e6'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:40.533377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] The token cache is being invalidate because project ddec3ec555a24bb59559b553d060d802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:31:40.533666 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:40.534419 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:40.535870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1375/2746] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:31:40 2022] DELETE /identity/v3/projects/ddec3ec555a24bb59559b553d060d802 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:40.547603 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8e942-5526-40b8-976c-f5f1ab8fd6a7 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:40.557736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8e942-5526-40b8-976c-f5f1ab8fd6a7 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:40.615783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:40.616423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:40.616702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:40.616973 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] PATH_INFO: `/v3/users/1417ec2fa06a433885bb8770c48a9b92` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:40.618167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1417ec2fa06a433885bb8770c48a9b92)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:40.620085 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:40.789155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1417ec2fa06a433885bb8770c48a9b92'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:40.889831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1417ec2fa06a433885bb8770c48a9b92'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:40.897652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1417ec2fa06a433885bb8770c48a9b92'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:40.910836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1417ec2fa06a433885bb8770c48a9b92'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:40.911238 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-1e932599-70b9-4809-8ff3-a169e9cba962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1417ec2fa06a433885bb8770c48a9b92'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:40.917261 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1372/2747] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:31:40 2022] DELETE /identity/v3/users/1417ec2fa06a433885bb8770c48a9b92 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:41.331037 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:41.332535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:41.346643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7711a-68eb-443d-bacb-089fca3860e6 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:41.408474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:41.409113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:41.409440 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:41.409749 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] PATH_INFO: `/v3/projects/204bdde2dde847bda042c23f9b3f3859` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:41.410895 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=204bdde2dde847bda042c23f9b3f3859)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:41.412757 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:41.576823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '204bdde2dde847bda042c23f9b3f3859', 'request_id': 'req-7431c269-6f2c-4d94-90bb-ec3f42e832ce'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:41.632168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '204bdde2dde847bda042c23f9b3f3859', 'request_id': 'req-7431c269-6f2c-4d94-90bb-ec3f42e832ce'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:41.639151 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '204bdde2dde847bda042c23f9b3f3859', 'request_id': 'req-7431c269-6f2c-4d94-90bb-ec3f42e832ce'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:41.645729 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:41.650552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] The token cache is being invalidate because project 204bdde2dde847bda042c23f9b3f3859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:31:41.650833 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:41.651462 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7431c269-6f2c-4d94-90bb-ec3f42e832ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:41.652915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1376/2748] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:31:41 2022] DELETE /identity/v3/projects/204bdde2dde847bda042c23f9b3f3859 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:41.657772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad6b13-fec5-4d5f-b936-08a18ec0d7da None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:41.658278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad6b13-fec5-4d5f-b936-08a18ec0d7da None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:41.658375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad6b13-fec5-4d5f-b936-08a18ec0d7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:41.679153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-08ad6b13-fec5-4d5f-b936-08a18ec0d7da None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:41.680631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-08ad6b13-fec5-4d5f-b936-08a18ec0d7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:41.685772 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:41.741455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1373/2749] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:31:41.750421 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb49b21-519a-4b8c-97e1-ad123fa81ebc None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:41.751717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb49b21-519a-4b8c-97e1-ad123fa81ebc None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:41.751717 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb49b21-519a-4b8c-97e1-ad123fa81ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:41.770967 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6cb49b21-519a-4b8c-97e1-ad123fa81ebc None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:41.772162 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6cb49b21-519a-4b8c-97e1-ad123fa81ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:41.780614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1377/2750] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:31:41.785429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad6b13-fec5-4d5f-b936-08a18ec0d7da None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:41.795595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad6b13-fec5-4d5f-b936-08a18ec0d7da None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:41.800294 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942b1ef-b22c-4fab-886f-1c4425104e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:41.800882 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942b1ef-b22c-4fab-886f-1c4425104e77 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:41.801157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942b1ef-b22c-4fab-886f-1c4425104e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:41.801425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942b1ef-b22c-4fab-886f-1c4425104e77 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:41.802173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942b1ef-b22c-4fab-886f-1c4425104e77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:31:41.802470 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942b1ef-b22c-4fab-886f-1c4425104e77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:41.804045 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f942b1ef-b22c-4fab-886f-1c4425104e77 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:41.816394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1374/2751] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:31:41.825259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb49b21-519a-4b8c-97e1-ad123fa81ebc None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:41.834856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb49b21-519a-4b8c-97e1-ad123fa81ebc None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:41.839853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e922603-5a4b-4cad-8a94-128aa659a6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:41.840445 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922603-5a4b-4cad-8a94-128aa659a6ad admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:41.841219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922603-5a4b-4cad-8a94-128aa659a6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:41.841543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922603-5a4b-4cad-8a94-128aa659a6ad admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:41.842200 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e922603-5a4b-4cad-8a94-128aa659a6ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:41.844115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e922603-5a4b-4cad-8a94-128aa659a6ad admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:41.929756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1378/2752] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:31:41 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:31:41.934285 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942b1ef-b22c-4fab-886f-1c4425104e77 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:41.944674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f942b1ef-b22c-4fab-886f-1c4425104e77 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:42.002303 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e051e39-23b1-485a-9444-5088692a0848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:42.002906 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e051e39-23b1-485a-9444-5088692a0848 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.003182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e051e39-23b1-485a-9444-5088692a0848 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.003450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e051e39-23b1-485a-9444-5088692a0848 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:42.004074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e051e39-23b1-485a-9444-5088692a0848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:42.006010 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e051e39-23b1-485a-9444-5088692a0848 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:42.115909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1375/2753] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:31:41 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:31:42.120088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e922603-5a4b-4cad-8a94-128aa659a6ad admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:42.130902 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e922603-5a4b-4cad-8a94-128aa659a6ad admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:42.135788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438ee58-67ac-4115-8217-c94c2ef88c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:42.136388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438ee58-67ac-4115-8217-c94c2ef88c0e admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.136669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438ee58-67ac-4115-8217-c94c2ef88c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.136941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438ee58-67ac-4115-8217-c94c2ef88c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:42.137584 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438ee58-67ac-4115-8217-c94c2ef88c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:42.139228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438ee58-67ac-4115-8217-c94c2ef88c0e admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:42.149827 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1379/2754] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:42.154486 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e051e39-23b1-485a-9444-5088692a0848 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:42.164769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e051e39-23b1-485a-9444-5088692a0848 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:42.170175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4d7f3-5431-43c8-a843-f1d892326e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:42.170764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4d7f3-5431-43c8-a843-f1d892326e11 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.171033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4d7f3-5431-43c8-a843-f1d892326e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.171298 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4d7f3-5431-43c8-a843-f1d892326e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:42.171926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4d7f3-5431-43c8-a843-f1d892326e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:42.173487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4d7f3-5431-43c8-a843-f1d892326e11 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:42.184586 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1376/2755] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:31:42.189114 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438ee58-67ac-4115-8217-c94c2ef88c0e admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:42.199512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438ee58-67ac-4115-8217-c94c2ef88c0e admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:42.204712 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9ce3b-8a38-4ea6-83ac-5e5a047c1683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:42.205354 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ce3b-8a38-4ea6-83ac-5e5a047c1683 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.205651 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ce3b-8a38-4ea6-83ac-5e5a047c1683 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.205921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9ce3b-8a38-4ea6-83ac-5e5a047c1683 admin admin] PATH_INFO: `/v3/projects/312db86ef4e84567b676f8e226b5dd8b/users/b6634d9233014381b0cb5953375ec869/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:42.221111 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9ce3b-8a38-4ea6-83ac-5e5a047c1683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=312db86ef4e84567b676f8e226b5dd8b, user_id=b6634d9233014381b0cb5953375ec869, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:42.223992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9ce3b-8a38-4ea6-83ac-5e5a047c1683 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:42.333475 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1380/2756] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:31:42 2022] PUT /identity/v3/projects/312db86ef4e84567b676f8e226b5dd8b/users/b6634d9233014381b0cb5953375ec869/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:42.345629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939115e-5174-408d-9e09-5581928eba8d None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.345900 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939115e-5174-408d-9e09-5581928eba8d None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.346191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939115e-5174-408d-9e09-5581928eba8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:42.374654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-6939115e-5174-408d-9e09-5581928eba8d None None] MFA Rules not processed for user `b6634d9233014381b0cb5953375ec869`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:42.378139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-6939115e-5174-408d-9e09-5581928eba8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:42.446484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1377/2757] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:42.451346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9ce3b-8a38-4ea6-83ac-5e5a047c1683 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:42.516659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9ce3b-8a38-4ea6-83ac-5e5a047c1683 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:42.521430 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23da5f-57ad-459f-bc05-6fb2117ff4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:42.522065 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23da5f-57ad-459f-bc05-6fb2117ff4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.522347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23da5f-57ad-459f-bc05-6fb2117ff4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.522616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23da5f-57ad-459f-bc05-6fb2117ff4e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:42.523232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23da5f-57ad-459f-bc05-6fb2117ff4e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:42.525137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23da5f-57ad-459f-bc05-6fb2117ff4e5 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:42.629265 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1381/2758] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:31:42 2022] POST /identity/v3/projects => generated 379 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:31:42.633770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939115e-5174-408d-9e09-5581928eba8d None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:42.641128 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:42.645778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939115e-5174-408d-9e09-5581928eba8d None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:42.704299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836103cc-7fff-4b13-8a86-6058e3e7eafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:42.704990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836103cc-7fff-4b13-8a86-6058e3e7eafa admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.705189 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836103cc-7fff-4b13-8a86-6058e3e7eafa admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.705480 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836103cc-7fff-4b13-8a86-6058e3e7eafa admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:42.706139 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836103cc-7fff-4b13-8a86-6058e3e7eafa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:42.708114 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836103cc-7fff-4b13-8a86-6058e3e7eafa admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:42.789681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1378/2759] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:31:42 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:31:42.793754 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23da5f-57ad-459f-bc05-6fb2117ff4e5 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:42.804727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23da5f-57ad-459f-bc05-6fb2117ff4e5 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:42.810523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cd096-bae0-4bed-82cf-5bba7903acdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:42.811154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cd096-bae0-4bed-82cf-5bba7903acdd admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.811432 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cd096-bae0-4bed-82cf-5bba7903acdd admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.811702 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cd096-bae0-4bed-82cf-5bba7903acdd admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:42.812340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cd096-bae0-4bed-82cf-5bba7903acdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:42.813910 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cd096-bae0-4bed-82cf-5bba7903acdd admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:42.832744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1382/2760] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:42.836810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836103cc-7fff-4b13-8a86-6058e3e7eafa admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:42.847131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836103cc-7fff-4b13-8a86-6058e3e7eafa admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:42.852034 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e61720-d702-4a33-ba97-197e004d3d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:42.852648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e61720-d702-4a33-ba97-197e004d3d0c admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.852918 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e61720-d702-4a33-ba97-197e004d3d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.853159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e61720-d702-4a33-ba97-197e004d3d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:42.854059 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e61720-d702-4a33-ba97-197e004d3d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:42.855398 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e61720-d702-4a33-ba97-197e004d3d0c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:42.869733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1379/2761] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:42.874557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cd096-bae0-4bed-82cf-5bba7903acdd admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:42.884992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cd096-bae0-4bed-82cf-5bba7903acdd admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:42.890644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c6761-bcf1-4b7c-80f8-9dfb183c5df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:42.891305 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c6761-bcf1-4b7c-80f8-9dfb183c5df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.891585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c6761-bcf1-4b7c-80f8-9dfb183c5df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.891856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c6761-bcf1-4b7c-80f8-9dfb183c5df3 admin admin] PATH_INFO: `/v3/projects/b58022af788142db9445de40af7e2927/users/8b352703a35a4236a99ae601085dda5b/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:42.907923 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c6761-bcf1-4b7c-80f8-9dfb183c5df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58022af788142db9445de40af7e2927, user_id=8b352703a35a4236a99ae601085dda5b, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:42.910579 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c6761-bcf1-4b7c-80f8-9dfb183c5df3 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:42.983559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1383/2762] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:31:42 2022] PUT /identity/v3/projects/b58022af788142db9445de40af7e2927/users/8b352703a35a4236a99ae601085dda5b/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:42.993768 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee46d4f-7f6e-4504-81bf-dbcf8b14d361 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:42.994232 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee46d4f-7f6e-4504-81bf-dbcf8b14d361 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:42.994498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee46d4f-7f6e-4504-81bf-dbcf8b14d361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:43.022848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-3ee46d4f-7f6e-4504-81bf-dbcf8b14d361 None None] MFA Rules not processed for user `8b352703a35a4236a99ae601085dda5b`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:43.024102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3ee46d4f-7f6e-4504-81bf-dbcf8b14d361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:43.098158 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1380/2763] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:31:43.103059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c6761-bcf1-4b7c-80f8-9dfb183c5df3 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:43.154083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:43.170873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c6761-bcf1-4b7c-80f8-9dfb183c5df3 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:43.176286 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b051c5-00df-47eb-9a70-0ae7d4cbdd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:43.176882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b051c5-00df-47eb-9a70-0ae7d4cbdd9c admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:43.177161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b051c5-00df-47eb-9a70-0ae7d4cbdd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:43.177429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b051c5-00df-47eb-9a70-0ae7d4cbdd9c admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:43.178090 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b051c5-00df-47eb-9a70-0ae7d4cbdd9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:43.180331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b051c5-00df-47eb-9a70-0ae7d4cbdd9c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:43.263260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1384/2764] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:31:43 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:31:43.267580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee46d4f-7f6e-4504-81bf-dbcf8b14d361 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:43.279407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee46d4f-7f6e-4504-81bf-dbcf8b14d361 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:43.341200 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8262e1-e5dd-491b-886d-187179d860b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:43.341812 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8262e1-e5dd-491b-886d-187179d860b7 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:43.342116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8262e1-e5dd-491b-886d-187179d860b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:43.342385 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8262e1-e5dd-491b-886d-187179d860b7 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:43.343011 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8262e1-e5dd-491b-886d-187179d860b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:43.344896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8262e1-e5dd-491b-886d-187179d860b7 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:43.461172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1381/2765] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:31:43 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:31:43.465143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b051c5-00df-47eb-9a70-0ae7d4cbdd9c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:43.475842 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b051c5-00df-47eb-9a70-0ae7d4cbdd9c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:43.480665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bf2f8-e668-4ccf-b0a1-da91a0aca165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:43.481259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bf2f8-e668-4ccf-b0a1-da91a0aca165 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:43.481538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bf2f8-e668-4ccf-b0a1-da91a0aca165 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:43.481807 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bf2f8-e668-4ccf-b0a1-da91a0aca165 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:43.482486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bf2f8-e668-4ccf-b0a1-da91a0aca165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:43.484055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bf2f8-e668-4ccf-b0a1-da91a0aca165 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:43.494331 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1385/2766] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:43.499945 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8262e1-e5dd-491b-886d-187179d860b7 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:43.505289 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:43.513931 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8262e1-e5dd-491b-886d-187179d860b7 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:43.519826 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e139703-8f08-40e4-a336-e40fc871c486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:43.520474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139703-8f08-40e4-a336-e40fc871c486 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:43.520749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139703-8f08-40e4-a336-e40fc871c486 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:43.521017 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e139703-8f08-40e4-a336-e40fc871c486 admin admin] PATH_INFO: `/v3/projects/ee57ae7458cb47ffae5d190320973a5f/users/d0e52007f87545f8a2dc1553a0ecfe03/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:43.536684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e139703-8f08-40e4-a336-e40fc871c486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee57ae7458cb47ffae5d190320973a5f, user_id=d0e52007f87545f8a2dc1553a0ecfe03, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:43.540509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e139703-8f08-40e4-a336-e40fc871c486 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:43.612605 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1382/2767] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:31:43 2022] PUT /identity/v3/projects/ee57ae7458cb47ffae5d190320973a5f/users/d0e52007f87545f8a2dc1553a0ecfe03/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:43.616671 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bf2f8-e668-4ccf-b0a1-da91a0aca165 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:43.628094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bf2f8-e668-4ccf-b0a1-da91a0aca165 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:43.689377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf3f76-69a9-4730-8d2c-b985088e42b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:43.689999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf3f76-69a9-4730-8d2c-b985088e42b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:43.690274 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf3f76-69a9-4730-8d2c-b985088e42b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:43.690539 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf3f76-69a9-4730-8d2c-b985088e42b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:43.691166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf3f76-69a9-4730-8d2c-b985088e42b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:43.692727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf3f76-69a9-4730-8d2c-b985088e42b8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:43.703372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1386/2768] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:43.707589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e139703-8f08-40e4-a336-e40fc871c486 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:43.718223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e139703-8f08-40e4-a336-e40fc871c486 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:43.722936 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eaa241-1497-4661-a17f-8da01eec6c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:43.723589 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eaa241-1497-4661-a17f-8da01eec6c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:43.723888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eaa241-1497-4661-a17f-8da01eec6c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:43.724157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eaa241-1497-4661-a17f-8da01eec6c8b admin admin] PATH_INFO: `/v3/projects/ee57ae7458cb47ffae5d190320973a5f/users/d0e52007f87545f8a2dc1553a0ecfe03/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:43.726236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eaa241-1497-4661-a17f-8da01eec6c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee57ae7458cb47ffae5d190320973a5f, user_id=d0e52007f87545f8a2dc1553a0ecfe03, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:43.728899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eaa241-1497-4661-a17f-8da01eec6c8b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:43.740075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1383/2769] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:31:43 2022] PUT /identity/v3/projects/ee57ae7458cb47ffae5d190320973a5f/users/d0e52007f87545f8a2dc1553a0ecfe03/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:43.752210 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429879a8-9183-46eb-8a09-ef2dc30a11fe None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:43.752586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429879a8-9183-46eb-8a09-ef2dc30a11fe None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:43.752937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429879a8-9183-46eb-8a09-ef2dc30a11fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:43.781891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-429879a8-9183-46eb-8a09-ef2dc30a11fe None None] MFA Rules not processed for user `d0e52007f87545f8a2dc1553a0ecfe03`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:43.783138 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-429879a8-9183-46eb-8a09-ef2dc30a11fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:43.854308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1387/2770] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:43.866155 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eaa241-1497-4661-a17f-8da01eec6c8b admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:43.867909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e2eaa241-1497-4661-a17f-8da01eec6c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:43.931546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eaa241-1497-4661-a17f-8da01eec6c8b admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:43.935785 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586e187-b2a4-4f27-b0dc-c84e91eddf97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:43.936379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586e187-b2a4-4f27-b0dc-c84e91eddf97 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:43.936655 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586e187-b2a4-4f27-b0dc-c84e91eddf97 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:43.936926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586e187-b2a4-4f27-b0dc-c84e91eddf97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:43.946493 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586e187-b2a4-4f27-b0dc-c84e91eddf97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:43.947939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f586e187-b2a4-4f27-b0dc-c84e91eddf97 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:43.957419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1384/2771] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:31:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:31:44.056715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429879a8-9183-46eb-8a09-ef2dc30a11fe None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:44.058167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-429879a8-9183-46eb-8a09-ef2dc30a11fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:44.071018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429879a8-9183-46eb-8a09-ef2dc30a11fe None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:44.074986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5c65a-719f-478b-86bb-1555611388b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:44.075646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5c65a-719f-478b-86bb-1555611388b9 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:44.075968 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5c65a-719f-478b-86bb-1555611388b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:44.076283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5c65a-719f-478b-86bb-1555611388b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:44.115721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:44.135043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5c65a-719f-478b-86bb-1555611388b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:44.136511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5c65a-719f-478b-86bb-1555611388b9 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:44.146098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1388/2772] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:31:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:31:48.429535 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586e187-b2a4-4f27-b0dc-c84e91eddf97 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:48.493265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586e187-b2a4-4f27-b0dc-c84e91eddf97 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:48.497728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:48.498379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:48.499239 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:48.499507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] PATH_INFO: `/v3/users/b6634d9233014381b0cb5953375ec869` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:48.500617 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6634d9233014381b0cb5953375ec869)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:48.502563 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:48.705746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6634d9233014381b0cb5953375ec869'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:48.803860 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6634d9233014381b0cb5953375ec869'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:48.811879 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6634d9233014381b0cb5953375ec869'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:48.817059 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:48.823779 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6634d9233014381b0cb5953375ec869'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:48.824455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6634d9233014381b0cb5953375ec869'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:48.830278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1385/2773] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:31:48 2022] DELETE /identity/v3/users/b6634d9233014381b0cb5953375ec869 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:48.834796 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0863d7-2003-4c34-a4dd-cf97dd975a3f None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:48.835262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0863d7-2003-4c34-a4dd-cf97dd975a3f None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:48.835379 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0863d7-2003-4c34-a4dd-cf97dd975a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:48.854960 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-ec0863d7-2003-4c34-a4dd-cf97dd975a3f None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:48.856152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ec0863d7-2003-4c34-a4dd-cf97dd975a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:48.914255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1389/2774] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:48.924759 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:48.926185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:48.984670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef1ea4-fee7-4c27-a20c-42925bf0ea3a admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:48.988632 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5f597-c9f3-43ec-ad76-52318f89fefe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:48.988899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5f597-c9f3-43ec-ad76-52318f89fefe service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:48.989175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5f597-c9f3-43ec-ad76-52318f89fefe service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:48.989445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5f597-c9f3-43ec-ad76-52318f89fefe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:48.998704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5f597-c9f3-43ec-ad76-52318f89fefe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:49.000176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5f597-c9f3-43ec-ad76-52318f89fefe service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:49.009595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1386/2775] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:31:49.537239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0863d7-2003-4c34-a4dd-cf97dd975a3f None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:49.538721 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-ec0863d7-2003-4c34-a4dd-cf97dd975a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:49.551127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0863d7-2003-4c34-a4dd-cf97dd975a3f None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:49.555978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:49.556602 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:49.556903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:49.557166 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] PATH_INFO: `/v3/projects/312db86ef4e84567b676f8e226b5dd8b` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:49.558313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=312db86ef4e84567b676f8e226b5dd8b)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:49.560181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:49.667480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '312db86ef4e84567b676f8e226b5dd8b', 'request_id': 'req-c532f533-db41-4bf1-9965-431af35ba7a2'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:49.761115 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '312db86ef4e84567b676f8e226b5dd8b', 'request_id': 'req-c532f533-db41-4bf1-9965-431af35ba7a2'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:49.768084 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '312db86ef4e84567b676f8e226b5dd8b', 'request_id': 'req-c532f533-db41-4bf1-9965-431af35ba7a2'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:49.778963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] The token cache is being invalidate because project 312db86ef4e84567b676f8e226b5dd8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:31:49.779182 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:49.779863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:49.781346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1390/2776] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:31:49 2022] DELETE /identity/v3/projects/312db86ef4e84567b676f8e226b5dd8b => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:49.785366 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5f597-c9f3-43ec-ad76-52318f89fefe service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:49.795113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5f597-c9f3-43ec-ad76-52318f89fefe service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:49.850608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:49.851281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:49.852636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:49.852636 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] PATH_INFO: `/v3/users/8b352703a35a4236a99ae601085dda5b` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:49.854697 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b352703a35a4236a99ae601085dda5b)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:49.857626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:50.101340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b352703a35a4236a99ae601085dda5b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:50.196444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b352703a35a4236a99ae601085dda5b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:50.205002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b352703a35a4236a99ae601085dda5b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:50.218088 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b352703a35a4236a99ae601085dda5b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:50.218831 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b352703a35a4236a99ae601085dda5b'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:50.224978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1387/2777] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:31:49 2022] DELETE /identity/v3/users/8b352703a35a4236a99ae601085dda5b => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:50.671547 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:50.673099 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:50.687805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532f533-db41-4bf1-9965-431af35ba7a2 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:50.745676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:50.746329 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:50.746607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:50.746874 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] PATH_INFO: `/v3/projects/b58022af788142db9445de40af7e2927` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:50.748003 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58022af788142db9445de40af7e2927)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:50.749981 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:50.752272 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:50.940748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58022af788142db9445de40af7e2927', 'request_id': 'req-af03449a-2b6e-490a-8da5-1275ed1c6e95'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:51.050087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58022af788142db9445de40af7e2927', 'request_id': 'req-af03449a-2b6e-490a-8da5-1275ed1c6e95'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:51.057207 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58022af788142db9445de40af7e2927', 'request_id': 'req-af03449a-2b6e-490a-8da5-1275ed1c6e95'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:51.068486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] The token cache is being invalidate because project b58022af788142db9445de40af7e2927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:31:51.068772 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:51.069418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:51.070936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1391/2778] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:31:50 2022] DELETE /identity/v3/projects/b58022af788142db9445de40af7e2927 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:51.075072 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:51.076496 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:51.090231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9c8ca-17b6-43d3-ab87-ec4fcbbe4bc0 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:51.095328 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:51.145840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:51.146517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:51.146792 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:51.147056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] PATH_INFO: `/v3/users/d0e52007f87545f8a2dc1553a0ecfe03` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:51.148172 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e52007f87545f8a2dc1553a0ecfe03)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:51.150281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:51.332179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e52007f87545f8a2dc1553a0ecfe03'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:51.404353 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e52007f87545f8a2dc1553a0ecfe03'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:51.412631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e52007f87545f8a2dc1553a0ecfe03'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:51.425522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e52007f87545f8a2dc1553a0ecfe03'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:51.426248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-791d299c-54d5-4273-92d5-93201657c595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e52007f87545f8a2dc1553a0ecfe03'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:51.432776 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1388/2779] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:31:51 2022] DELETE /identity/v3/users/d0e52007f87545f8a2dc1553a0ecfe03 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:51.912204 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:51.913672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:51.927094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03449a-2b6e-490a-8da5-1275ed1c6e95 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:51.986326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:51.986952 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:51.987225 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:51.987493 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] PATH_INFO: `/v3/projects/ee57ae7458cb47ffae5d190320973a5f` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:51.988598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee57ae7458cb47ffae5d190320973a5f)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:51.990502 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:52.097108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee57ae7458cb47ffae5d190320973a5f', 'request_id': 'req-c3dbcabe-338b-402b-a380-da8d1f19c60b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:52.207016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee57ae7458cb47ffae5d190320973a5f', 'request_id': 'req-c3dbcabe-338b-402b-a380-da8d1f19c60b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:52.214015 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee57ae7458cb47ffae5d190320973a5f', 'request_id': 'req-c3dbcabe-338b-402b-a380-da8d1f19c60b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:52.224920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] The token cache is being invalidate because project ee57ae7458cb47ffae5d190320973a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:31:52.225197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:52.225832 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-c3dbcabe-338b-402b-a380-da8d1f19c60b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:31:52.227318 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1392/2780] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:31:51 2022] DELETE /identity/v3/projects/ee57ae7458cb47ffae5d190320973a5f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:52.234030 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699e821-b364-4212-9ec4-ec7274e62776 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:52.234543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699e821-b364-4212-9ec4-ec7274e62776 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:52.234644 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699e821-b364-4212-9ec4-ec7274e62776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:52.253392 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8699e821-b364-4212-9ec4-ec7274e62776 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:52.254626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8699e821-b364-4212-9ec4-ec7274e62776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:52.319062 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1389/2781] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:52.327788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad88eed-0c48-400c-ac93-cc92bf6923c3 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:52.331810 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad88eed-0c48-400c-ac93-cc92bf6923c3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:52.331810 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad88eed-0c48-400c-ac93-cc92bf6923c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:52.350363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-aad88eed-0c48-400c-ac93-cc92bf6923c3 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:52.351574 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-aad88eed-0c48-400c-ac93-cc92bf6923c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:52.359944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1393/2782] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:52.364387 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699e821-b364-4212-9ec4-ec7274e62776 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:52.374098 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699e821-b364-4212-9ec4-ec7274e62776 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:52.379087 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46530490-ff96-47b5-93c5-58e043842c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:52.379668 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46530490-ff96-47b5-93c5-58e043842c0e admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:52.379941 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46530490-ff96-47b5-93c5-58e043842c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:52.380206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46530490-ff96-47b5-93c5-58e043842c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:52.380840 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46530490-ff96-47b5-93c5-58e043842c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:31:52.381132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46530490-ff96-47b5-93c5-58e043842c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:52.382723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46530490-ff96-47b5-93c5-58e043842c0e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:52.394845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1390/2783] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:31:52.403823 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad88eed-0c48-400c-ac93-cc92bf6923c3 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:52.412963 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad88eed-0c48-400c-ac93-cc92bf6923c3 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:52.417818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713a275-98cd-469b-8975-e308311175aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:52.418441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713a275-98cd-469b-8975-e308311175aa admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:52.418716 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713a275-98cd-469b-8975-e308311175aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:52.418982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713a275-98cd-469b-8975-e308311175aa admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:52.419627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713a275-98cd-469b-8975-e308311175aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:52.421554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713a275-98cd-469b-8975-e308311175aa admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:52.436198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:52.579256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1394/2784] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:31:52 2022] POST /identity/v3/projects => generated 369 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:31:52.583206 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46530490-ff96-47b5-93c5-58e043842c0e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:52.593170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46530490-ff96-47b5-93c5-58e043842c0e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:52.606283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:52.652015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58bfca-2456-4536-8784-6f400889b5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:52.652608 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58bfca-2456-4536-8784-6f400889b5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:52.652899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58bfca-2456-4536-8784-6f400889b5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:52.653150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58bfca-2456-4536-8784-6f400889b5d3 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:52.653857 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58bfca-2456-4536-8784-6f400889b5d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:52.655771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58bfca-2456-4536-8784-6f400889b5d3 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:52.744012 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1391/2785] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:31:52 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:31:52.748324 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713a275-98cd-469b-8975-e308311175aa admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:52.759556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713a275-98cd-469b-8975-e308311175aa admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:52.764173 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b335c5-3803-468b-8de8-36ca73ad552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:52.764766 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b335c5-3803-468b-8de8-36ca73ad552b admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:52.765075 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b335c5-3803-468b-8de8-36ca73ad552b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:52.765373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b335c5-3803-468b-8de8-36ca73ad552b admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:52.766070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b335c5-3803-468b-8de8-36ca73ad552b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:52.767628 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b335c5-3803-468b-8de8-36ca73ad552b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:52.777878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1395/2786] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:52.781771 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58bfca-2456-4536-8784-6f400889b5d3 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:52.791804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58bfca-2456-4536-8784-6f400889b5d3 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:52.796249 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6255002-a28f-466d-86e5-95e1c53b0e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:52.797357 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6255002-a28f-466d-86e5-95e1c53b0e39 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:52.797634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6255002-a28f-466d-86e5-95e1c53b0e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:52.797902 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6255002-a28f-466d-86e5-95e1c53b0e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:52.798595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6255002-a28f-466d-86e5-95e1c53b0e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:52.800164 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6255002-a28f-466d-86e5-95e1c53b0e39 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:52.811157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1392/2787] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:31:52.815972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b335c5-3803-468b-8de8-36ca73ad552b admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:52.826183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b335c5-3803-468b-8de8-36ca73ad552b admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:52.830889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d783a-7063-41f1-a3a3-96e5cf8a6417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:52.831533 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d783a-7063-41f1-a3a3-96e5cf8a6417 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:52.831805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d783a-7063-41f1-a3a3-96e5cf8a6417 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:52.832073 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d783a-7063-41f1-a3a3-96e5cf8a6417 admin admin] PATH_INFO: `/v3/projects/6a95bae4c1004bafab21d682be008d11/users/7a3fa2395eb84a11b2d6ee5644575c56/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:52.848659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d783a-7063-41f1-a3a3-96e5cf8a6417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a95bae4c1004bafab21d682be008d11, user_id=7a3fa2395eb84a11b2d6ee5644575c56, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:52.851662 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d783a-7063-41f1-a3a3-96e5cf8a6417 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:52.965008 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1396/2788] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:31:52 2022] PUT /identity/v3/projects/6a95bae4c1004bafab21d682be008d11/users/7a3fa2395eb84a11b2d6ee5644575c56/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:31:52.976371 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c36a5-e46e-4a6f-9c1b-a908e0b4c6c8 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:52.976736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c36a5-e46e-4a6f-9c1b-a908e0b4c6c8 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:52.977386 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c36a5-e46e-4a6f-9c1b-a908e0b4c6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:53.005383 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-2a6c36a5-e46e-4a6f-9c1b-a908e0b4c6c8 None None] MFA Rules not processed for user `7a3fa2395eb84a11b2d6ee5644575c56`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:31:53.006637 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2a6c36a5-e46e-4a6f-9c1b-a908e0b4c6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:53.070845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1393/2789] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:31:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:31:53.082849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d783a-7063-41f1-a3a3-96e5cf8a6417 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:53.084694 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-237d783a-7063-41f1-a3a3-96e5cf8a6417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:53.141093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d783a-7063-41f1-a3a3-96e5cf8a6417 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:53.144826 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720184c-6e15-4b55-94fd-845d42bf1c62 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33003a9bf4454f5bac2e6ad73544efc4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:53.145411 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720184c-6e15-4b55-94fd-845d42bf1c62 service glance] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:53.145685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720184c-6e15-4b55-94fd-845d42bf1c62 service glance] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:53.145974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720184c-6e15-4b55-94fd-845d42bf1c62 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:53.154417 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720184c-6e15-4b55-94fd-845d42bf1c62 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:53.155812 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720184c-6e15-4b55-94fd-845d42bf1c62 service glance] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:53.164998 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1397/2790] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:31:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:31:55.621721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c36a5-e46e-4a6f-9c1b-a908e0b4c6c8 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:31:55.623273 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2a6c36a5-e46e-4a6f-9c1b-a908e0b4c6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:55.686851 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c36a5-e46e-4a6f-9c1b-a908e0b4c6c8 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:31:55.692555 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee0616-8035-48ec-8186-12dd2462cf0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:31:55.693163 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee0616-8035-48ec-8186-12dd2462cf0b service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:31:55.693435 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee0616-8035-48ec-8186-12dd2462cf0b service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:31:55.693702 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee0616-8035-48ec-8186-12dd2462cf0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:31:55.695468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9cee0616-8035-48ec-8186-12dd2462cf0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:31:55.738136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:31:55.788922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee0616-8035-48ec-8186-12dd2462cf0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:31:55.788922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee0616-8035-48ec-8186-12dd2462cf0b service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:31:55.807884 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1394/2791] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:31:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:32:03.161032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:32:05.807629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=23 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:32:54.941027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720184c-6e15-4b55-94fd-845d42bf1c62 service glance] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:32:55.016861 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720184c-6e15-4b55-94fd-845d42bf1c62 service glance] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:32:55.024646 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:32:55.025542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:32:55.025542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:32:55.026372 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] PATH_INFO: `/v3/users/7a3fa2395eb84a11b2d6ee5644575c56` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:32:55.027108 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3fa2395eb84a11b2d6ee5644575c56)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:32:55.029132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:32:55.108037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee0616-8035-48ec-8186-12dd2462cf0b service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:32:55.109583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9cee0616-8035-48ec-8186-12dd2462cf0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:32:55.166408 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee0616-8035-48ec-8186-12dd2462cf0b service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:32:55.169929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9932858-c11f-4532-bcb5-3392c640ab3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:32:55.170540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9932858-c11f-4532-bcb5-3392c640ab3c service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:32:55.170810 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9932858-c11f-4532-bcb5-3392c640ab3c service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:32:55.171076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9932858-c11f-4532-bcb5-3392c640ab3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:32:55.172868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e9932858-c11f-4532-bcb5-3392c640ab3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:32:55.185854 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9932858-c11f-4532-bcb5-3392c640ab3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:32:55.187305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9932858-c11f-4532-bcb5-3392c640ab3c service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:32:55.196778 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1395/2792] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:32:55.266141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3fa2395eb84a11b2d6ee5644575c56'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:32:55.354364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3fa2395eb84a11b2d6ee5644575c56'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:32:55.369941 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3fa2395eb84a11b2d6ee5644575c56'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:32:55.370787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3fa2395eb84a11b2d6ee5644575c56'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:32:55.371480 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3fa2395eb84a11b2d6ee5644575c56'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:32:55.391939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1398/2793] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:32:54 2022] DELETE /identity/v3/users/7a3fa2395eb84a11b2d6ee5644575c56 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:32:55.396985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75b2f5-a756-465a-a35a-e6f231870573 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:32:55.397275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75b2f5-a756-465a-a35a-e6f231870573 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:32:55.397568 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75b2f5-a756-465a-a35a-e6f231870573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:32:55.418217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-9e75b2f5-a756-465a-a35a-e6f231870573 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:32:55.419254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9e75b2f5-a756-465a-a35a-e6f231870573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:32:55.510502 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1396/2794] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:32:55.524113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:32:55.525652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:32:55.608450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f29afb-7761-462b-9b88-ab3bb16b5dcd admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:32:55.614013 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7d5ef-9a88-4a4b-a672-bcf2ba545aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:32:55.614819 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7d5ef-9a88-4a4b-a672-bcf2ba545aaf service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:32:55.615124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7d5ef-9a88-4a4b-a672-bcf2ba545aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:32:55.615397 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e7d5ef-9a88-4a4b-a672-bcf2ba545aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:32:55.626150 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7d5ef-9a88-4a4b-a672-bcf2ba545aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:32:55.627825 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e7d5ef-9a88-4a4b-a672-bcf2ba545aaf service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:32:55.639586 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1399/2795] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:32:56.113685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75b2f5-a756-465a-a35a-e6f231870573 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:32:56.115867 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9e75b2f5-a756-465a-a35a-e6f231870573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:32:56.137391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75b2f5-a756-465a-a35a-e6f231870573 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:32:56.143473 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:32:56.144202 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:32:56.144419 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:32:56.144615 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] PATH_INFO: `/v3/projects/6a95bae4c1004bafab21d682be008d11` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:32:56.145725 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a95bae4c1004bafab21d682be008d11)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:32:56.147681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:32:56.345841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a95bae4c1004bafab21d682be008d11', 'request_id': 'req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:32:56.418169 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a95bae4c1004bafab21d682be008d11', 'request_id': 'req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:32:56.427799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a95bae4c1004bafab21d682be008d11', 'request_id': 'req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:32:56.473887 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] The token cache is being invalidate because project 6a95bae4c1004bafab21d682be008d11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:32:56.475507 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:32:56.477449 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:32:56.482170 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1397/2796] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:32:56 2022] DELETE /identity/v3/projects/6a95bae4c1004bafab21d682be008d11 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:32:56.687247 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7d5ef-9a88-4a4b-a672-bcf2ba545aaf service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:32:56.713543 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e7d5ef-9a88-4a4b-a672-bcf2ba545aaf service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:32:56.729847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:32:56.784841 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d13e83-c65e-4690-a363-56191dfc5da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:32:56.785469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d13e83-c65e-4690-a363-56191dfc5da4 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:32:56.785753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d13e83-c65e-4690-a363-56191dfc5da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:32:56.786060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d13e83-c65e-4690-a363-56191dfc5da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:32:56.787882 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c5d13e83-c65e-4690-a363-56191dfc5da4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:32:56.865069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d13e83-c65e-4690-a363-56191dfc5da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:32:56.866554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d13e83-c65e-4690-a363-56191dfc5da4 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:32:56.878033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1400/2797] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:32:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:32:59.653227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:32:59.654974 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:32:59.673052 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa78a25-ff4d-43aa-948f-f2b7436bbaf0 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:32:59.680523 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:32:59.681146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:32:59.681421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:32:59.681685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] PATH_INFO: `/v3/users/ccad0b7ad5274d02b43f9c2fb857452a` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:32:59.682836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccad0b7ad5274d02b43f9c2fb857452a)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:32:59.684845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:32:59.711519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:32:59.962991 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccad0b7ad5274d02b43f9c2fb857452a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:00.067796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccad0b7ad5274d02b43f9c2fb857452a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:00.081684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccad0b7ad5274d02b43f9c2fb857452a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:00.111184 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccad0b7ad5274d02b43f9c2fb857452a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:00.113970 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-9476b444-0d62-4011-8bfd-722a32143e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccad0b7ad5274d02b43f9c2fb857452a'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:00.124798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1398/2798] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:32:59 2022] DELETE /identity/v3/users/ccad0b7ad5274d02b43f9c2fb857452a => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:00.612995 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d13e83-c65e-4690-a363-56191dfc5da4 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:00.646858 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d13e83-c65e-4690-a363-56191dfc5da4 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:00.734387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:00.735294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:00.735689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:00.736169 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] PATH_INFO: `/v3/projects/b44445ab9759401083af9109bfb04fed` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:00.737659 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44445ab9759401083af9109bfb04fed)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:00.739834 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:00.876307 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44445ab9759401083af9109bfb04fed', 'request_id': 'req-4f56d140-e94b-4188-9824-795585d33ea6'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:00.978649 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44445ab9759401083af9109bfb04fed', 'request_id': 'req-4f56d140-e94b-4188-9824-795585d33ea6'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:00.987515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44445ab9759401083af9109bfb04fed', 'request_id': 'req-4f56d140-e94b-4188-9824-795585d33ea6'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:00.993357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:01.001695 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] The token cache is being invalidate because project b44445ab9759401083af9109bfb04fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:33:01.002441 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:01.002876 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-4f56d140-e94b-4188-9824-795585d33ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:01.004707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1401/2799] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:33:00 2022] DELETE /identity/v3/projects/b44445ab9759401083af9109bfb04fed => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:01.011192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21e3b4-b43b-475d-8cee-38f37dda2bc4 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:01.011891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21e3b4-b43b-475d-8cee-38f37dda2bc4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:01.011891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac21e3b4-b43b-475d-8cee-38f37dda2bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:01.042904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ac21e3b4-b43b-475d-8cee-38f37dda2bc4 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:01.044954 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ac21e3b4-b43b-475d-8cee-38f37dda2bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:01.121988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1399/2800] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:33:01.130392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eca23c-8244-41b8-88f4-c1c94c0f2fd3 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:01.130676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eca23c-8244-41b8-88f4-c1c94c0f2fd3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:01.130952 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eca23c-8244-41b8-88f4-c1c94c0f2fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:01.151652 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-43eca23c-8244-41b8-88f4-c1c94c0f2fd3 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:01.152873 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-43eca23c-8244-41b8-88f4-c1c94c0f2fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:01.161317 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1402/2801] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:01.167378 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21e3b4-b43b-475d-8cee-38f37dda2bc4 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:01.177291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac21e3b4-b43b-475d-8cee-38f37dda2bc4 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:01.184151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e360d4-5f1a-4317-9e4b-5951d6ff2b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:01.184151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e360d4-5f1a-4317-9e4b-5951d6ff2b7e admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:01.184151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e360d4-5f1a-4317-9e4b-5951d6ff2b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:01.184151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e360d4-5f1a-4317-9e4b-5951d6ff2b7e admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:01.185198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e360d4-5f1a-4317-9e4b-5951d6ff2b7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:33:01.185198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e360d4-5f1a-4317-9e4b-5951d6ff2b7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:01.186272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e360d4-5f1a-4317-9e4b-5951d6ff2b7e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:01.199021 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1400/2802] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:33:01.203424 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eca23c-8244-41b8-88f4-c1c94c0f2fd3 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:01.221353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eca23c-8244-41b8-88f4-c1c94c0f2fd3 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:01.227083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98b243-7d5c-4f61-a697-518f23046b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:01.227896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98b243-7d5c-4f61-a697-518f23046b3b admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:01.228254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98b243-7d5c-4f61-a697-518f23046b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:01.228647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98b243-7d5c-4f61-a697-518f23046b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:01.229460 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98b243-7d5c-4f61-a697-518f23046b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:01.232495 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98b243-7d5c-4f61-a697-518f23046b3b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:01.396474 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1403/2803] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:33:01 2022] POST /identity/v3/projects => generated 371 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:33:01.401191 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e360d4-5f1a-4317-9e4b-5951d6ff2b7e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:01.414737 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e360d4-5f1a-4317-9e4b-5951d6ff2b7e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:01.493526 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279727ec-8735-4220-a9b3-6c67674b2f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:01.494162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279727ec-8735-4220-a9b3-6c67674b2f27 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:01.494436 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279727ec-8735-4220-a9b3-6c67674b2f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:01.494717 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279727ec-8735-4220-a9b3-6c67674b2f27 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:01.495344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279727ec-8735-4220-a9b3-6c67674b2f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:01.497425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279727ec-8735-4220-a9b3-6c67674b2f27 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:01.601119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1401/2804] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:33:01 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:01.605744 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98b243-7d5c-4f61-a697-518f23046b3b admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:01.618293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98b243-7d5c-4f61-a697-518f23046b3b admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:01.623589 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b98276-1e62-4f08-9157-a2ac087d4c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:01.624198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b98276-1e62-4f08-9157-a2ac087d4c5a admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:01.624468 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b98276-1e62-4f08-9157-a2ac087d4c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:01.624939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b98276-1e62-4f08-9157-a2ac087d4c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:01.625670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b98276-1e62-4f08-9157-a2ac087d4c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:01.627359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b98276-1e62-4f08-9157-a2ac087d4c5a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:01.639292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1404/2805] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:01.644933 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279727ec-8735-4220-a9b3-6c67674b2f27 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:01.655920 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279727ec-8735-4220-a9b3-6c67674b2f27 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:01.660899 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eff5d4-3a3e-44f6-bdd1-7819e62b3a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:01.661509 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eff5d4-3a3e-44f6-bdd1-7819e62b3a58 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:01.661868 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eff5d4-3a3e-44f6-bdd1-7819e62b3a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:01.662074 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eff5d4-3a3e-44f6-bdd1-7819e62b3a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:01.662736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eff5d4-3a3e-44f6-bdd1-7819e62b3a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:01.664666 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eff5d4-3a3e-44f6-bdd1-7819e62b3a58 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:01.676223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1402/2806] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:01.685266 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b98276-1e62-4f08-9157-a2ac087d4c5a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:01.696512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b98276-1e62-4f08-9157-a2ac087d4c5a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:01.701043 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71237d46-2813-49d4-b147-e68f8892178a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:01.701843 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71237d46-2813-49d4-b147-e68f8892178a admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:01.702153 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71237d46-2813-49d4-b147-e68f8892178a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:01.702418 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71237d46-2813-49d4-b147-e68f8892178a admin admin] PATH_INFO: `/v3/projects/8e1e22d5eb9f4d10976613f008501e1a/users/f97bbae5c7be4b62ae2d564d4ddc09b3/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:01.723206 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71237d46-2813-49d4-b147-e68f8892178a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1e22d5eb9f4d10976613f008501e1a, user_id=f97bbae5c7be4b62ae2d564d4ddc09b3, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:01.726611 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71237d46-2813-49d4-b147-e68f8892178a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:01.812907 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1405/2807] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:01 2022] PUT /identity/v3/projects/8e1e22d5eb9f4d10976613f008501e1a/users/f97bbae5c7be4b62ae2d564d4ddc09b3/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:33:01.825780 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7660d6e-f89c-46c6-bee6-97cfa7ce207c None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:01.826096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7660d6e-f89c-46c6-bee6-97cfa7ce207c None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:01.826369 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7660d6e-f89c-46c6-bee6-97cfa7ce207c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:01.835418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:01.859140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-d7660d6e-f89c-46c6-bee6-97cfa7ce207c None None] MFA Rules not processed for user `f97bbae5c7be4b62ae2d564d4ddc09b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:01.860522 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d7660d6e-f89c-46c6-bee6-97cfa7ce207c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:01.934283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1403/2808] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:01.948870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71237d46-2813-49d4-b147-e68f8892178a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:01.951093 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-71237d46-2813-49d4-b147-e68f8892178a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:02.057278 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71237d46-2813-49d4-b147-e68f8892178a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:02.066577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab00c0-e696-40b3-976f-00918a089894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:02.066577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab00c0-e696-40b3-976f-00918a089894 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:02.066577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab00c0-e696-40b3-976f-00918a089894 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:02.067061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab00c0-e696-40b3-976f-00918a089894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:02.075437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab00c0-e696-40b3-976f-00918a089894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:02.075437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ab00c0-e696-40b3-976f-00918a089894 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:02.086838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1406/2809] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:02.113084 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7660d6e-f89c-46c6-bee6-97cfa7ce207c None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:02.114685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d7660d6e-f89c-46c6-bee6-97cfa7ce207c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:02.190032 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7660d6e-f89c-46c6-bee6-97cfa7ce207c None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:02.194798 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa12e3-0d72-4e17-849c-a85fd73bd62a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:02.195403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa12e3-0d72-4e17-849c-a85fd73bd62a service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:02.195709 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa12e3-0d72-4e17-849c-a85fd73bd62a service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:02.195971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa12e3-0d72-4e17-849c-a85fd73bd62a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:02.197734 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c8aa12e3-0d72-4e17-849c-a85fd73bd62a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:02.214220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa12e3-0d72-4e17-849c-a85fd73bd62a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:02.214711 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa12e3-0d72-4e17-849c-a85fd73bd62a service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:02.228525 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1404/2810] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:33:05.205123 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab00c0-e696-40b3-976f-00918a089894 service nova] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:05.206760 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f9ab00c0-e696-40b3-976f-00918a089894 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:05.278706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab00c0-e696-40b3-976f-00918a089894 service nova] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:05.283577 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:05.284132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:05.284565 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:05.288580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] PATH_INFO: `/v3/users/99a24283bfde46b0872b1c4a10e8d617` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:05.288580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=99a24283bfde46b0872b1c4a10e8d617)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:05.288580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:05.289914 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:05.518713 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99a24283bfde46b0872b1c4a10e8d617'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:05.617544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99a24283bfde46b0872b1c4a10e8d617'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:05.631524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99a24283bfde46b0872b1c4a10e8d617'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:05.632309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99a24283bfde46b0872b1c4a10e8d617'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:05.634776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99a24283bfde46b0872b1c4a10e8d617'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:05.646070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1407/2811] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:33:05 2022] DELETE /identity/v3/users/99a24283bfde46b0872b1c4a10e8d617 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:05.918174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa12e3-0d72-4e17-849c-a85fd73bd62a service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:05.930504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa12e3-0d72-4e17-849c-a85fd73bd62a service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:06.000926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:06.001505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:06.001783 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:06.002147 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] PATH_INFO: `/v3/projects/7ee36e36177140ccb933e3d6f4f46b3c` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:06.003377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ee36e36177140ccb933e3d6f4f46b3c)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:06.005390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:06.107089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:06.120415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ee36e36177140ccb933e3d6f4f46b3c', 'request_id': 'req-374743b7-8b85-4628-a497-322a196a58ba'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.193627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ee36e36177140ccb933e3d6f4f46b3c', 'request_id': 'req-374743b7-8b85-4628-a497-322a196a58ba'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.201967 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ee36e36177140ccb933e3d6f4f46b3c', 'request_id': 'req-374743b7-8b85-4628-a497-322a196a58ba'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.213217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] The token cache is being invalidate because project 7ee36e36177140ccb933e3d6f4f46b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:33:06.213505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.214217 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.215724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1405/2812] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:33:05 2022] DELETE /identity/v3/projects/7ee36e36177140ccb933e3d6f4f46b3c => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:06.219847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:06.221285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:06.235369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dc7cf-ba7a-4166-9ba5-0964352c5ead admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:06.292364 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:06.292891 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:06.293163 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:06.293427 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] PATH_INFO: `/v3/users/ff544aeee2e14b32807ccfbdd03f8fb9` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:06.308194 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff544aeee2e14b32807ccfbdd03f8fb9)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:06.311940 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:06.521943 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff544aeee2e14b32807ccfbdd03f8fb9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.614425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:06.614425 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:06.623536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374743b7-8b85-4628-a497-322a196a58ba admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:06.629308 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff544aeee2e14b32807ccfbdd03f8fb9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.629468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:06.629983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:06.630312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:06.634582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] PATH_INFO: `/v3/users/f97bbae5c7be4b62ae2d564d4ddc09b3` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:06.634582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f97bbae5c7be4b62ae2d564d4ddc09b3)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:06.634582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:06.641256 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:06.642499 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff544aeee2e14b32807ccfbdd03f8fb9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.643276 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff544aeee2e14b32807ccfbdd03f8fb9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.643901 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff544aeee2e14b32807ccfbdd03f8fb9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.658526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1408/2813] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:33:06 2022] DELETE /identity/v3/users/ff544aeee2e14b32807ccfbdd03f8fb9 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:06.849662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f97bbae5c7be4b62ae2d564d4ddc09b3'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.962556 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:06.964157 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:06.985305 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f97bbae5c7be4b62ae2d564d4ddc09b3'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:06.999001 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f97bbae5c7be4b62ae2d564d4ddc09b3'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:07.014997 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f97bbae5c7be4b62ae2d564d4ddc09b3'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:07.015724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-03ee9922-3d69-45ac-8f36-c798bc9f5b92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f97bbae5c7be4b62ae2d564d4ddc09b3'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:07.022157 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1406/2814] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:33:06 2022] DELETE /identity/v3/users/f97bbae5c7be4b62ae2d564d4ddc09b3 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:33:07.027024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d8d4b-9a20-4c97-8620-d56ed00f2394 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:07.027313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d8d4b-9a20-4c97-8620-d56ed00f2394 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:07.027590 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d8d4b-9a20-4c97-8620-d56ed00f2394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:07.048499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-bd4d8d4b-9a20-4c97-8620-d56ed00f2394 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:07.049821 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bd4d8d4b-9a20-4c97-8620-d56ed00f2394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:07.057889 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea4f20-5107-4eeb-a55d-9a73b0c89165 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:07.113786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1407/2815] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:07.124431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:07.125051 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:07.125327 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d8d4b-9a20-4c97-8620-d56ed00f2394 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:07.125513 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:07.125736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] PATH_INFO: `/v3/projects/4cb7628373d148baa468e2cabfd9fccb` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:07.126782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bd4d8d4b-9a20-4c97-8620-d56ed00f2394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:07.137689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb7628373d148baa468e2cabfd9fccb)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:07.139668 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:07.171775 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:07.195602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d8d4b-9a20-4c97-8620-d56ed00f2394 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:07.199849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e657c-3109-4ef6-92e5-3e96c21bf3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:07.200442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e657c-3109-4ef6-92e5-3e96c21bf3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:07.200714 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e657c-3109-4ef6-92e5-3e96c21bf3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:07.201006 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e657c-3109-4ef6-92e5-3e96c21bf3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:07.210478 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e657c-3109-4ef6-92e5-3e96c21bf3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:07.212574 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e657c-3109-4ef6-92e5-3e96c21bf3c0 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:07.222178 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1408/2816] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:07.309358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb7628373d148baa468e2cabfd9fccb', 'request_id': 'req-7ad3b02e-91b6-4c23-8c82-2a316e32a022'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:07.473937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb7628373d148baa468e2cabfd9fccb', 'request_id': 'req-7ad3b02e-91b6-4c23-8c82-2a316e32a022'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:07.481848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb7628373d148baa468e2cabfd9fccb', 'request_id': 'req-7ad3b02e-91b6-4c23-8c82-2a316e32a022'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:07.493679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] The token cache is being invalidate because project 4cb7628373d148baa468e2cabfd9fccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:33:07.494155 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:07.494679 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-7ad3b02e-91b6-4c23-8c82-2a316e32a022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:07.496218 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1409/2817] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:33:06 2022] DELETE /identity/v3/projects/4cb7628373d148baa468e2cabfd9fccb => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:07.779595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e657c-3109-4ef6-92e5-3e96c21bf3c0 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:07.781173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8b7e657c-3109-4ef6-92e5-3e96c21bf3c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:07.795077 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e657c-3109-4ef6-92e5-3e96c21bf3c0 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:07.861853 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:07.862514 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:07.862791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:07.863066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] PATH_INFO: `/v3/projects/8e1e22d5eb9f4d10976613f008501e1a` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:07.864321 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e1e22d5eb9f4d10976613f008501e1a)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:07.866252 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:08.020024 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e1e22d5eb9f4d10976613f008501e1a', 'request_id': 'req-01605230-9690-463d-9987-6bb9e173b1b9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:08.109171 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e1e22d5eb9f4d10976613f008501e1a', 'request_id': 'req-01605230-9690-463d-9987-6bb9e173b1b9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:08.116373 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e1e22d5eb9f4d10976613f008501e1a', 'request_id': 'req-01605230-9690-463d-9987-6bb9e173b1b9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:08.127804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] The token cache is being invalidate because project 8e1e22d5eb9f4d10976613f008501e1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86843) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:33:08.128093 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:08.128760 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-01605230-9690-463d-9987-6bb9e173b1b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:08.130281 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1409/2818] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:33:07 2022] DELETE /identity/v3/projects/8e1e22d5eb9f4d10976613f008501e1a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:08.135220 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23d9d6-0a91-4cf8-9f93-785f37052adf None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:08.135951 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23d9d6-0a91-4cf8-9f93-785f37052adf None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:08.135951 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23d9d6-0a91-4cf8-9f93-785f37052adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:08.155233 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-be23d9d6-0a91-4cf8-9f93-785f37052adf None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:08.156526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-be23d9d6-0a91-4cf8-9f93-785f37052adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:08.166390 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:08.218477 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1410/2819] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:33:08.226234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9a9d-3866-46db-9485-c9caf00dd391 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:08.226521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9a9d-3866-46db-9485-c9caf00dd391 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:08.226793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f9a9d-3866-46db-9485-c9caf00dd391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:08.247743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-bf3f9a9d-3866-46db-9485-c9caf00dd391 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:08.249121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-bf3f9a9d-3866-46db-9485-c9caf00dd391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:08.257324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1410/2820] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:08.263748 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23d9d6-0a91-4cf8-9f93-785f37052adf None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:08.272415 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23d9d6-0a91-4cf8-9f93-785f37052adf None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:08.277616 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b66770-9097-4d34-b6b3-8d86e2ec4335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:08.278262 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b66770-9097-4d34-b6b3-8d86e2ec4335 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:08.278686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b66770-9097-4d34-b6b3-8d86e2ec4335 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:08.278962 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b66770-9097-4d34-b6b3-8d86e2ec4335 admin admin] PATH_INFO: `/v3/domains` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:08.279607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b66770-9097-4d34-b6b3-8d86e2ec4335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86844) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:33:08.279903 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b66770-9097-4d34-b6b3-8d86e2ec4335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:08.281481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b66770-9097-4d34-b6b3-8d86e2ec4335 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:08.295058 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1411/2821] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:33:08.300175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f9a9d-3866-46db-9485-c9caf00dd391 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:08.310723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f9a9d-3866-46db-9485-c9caf00dd391 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:08.315661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866c6b1-e071-419c-8769-e4d3e4808d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:08.316265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866c6b1-e071-419c-8769-e4d3e4808d76 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:08.316544 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866c6b1-e071-419c-8769-e4d3e4808d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:08.316834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866c6b1-e071-419c-8769-e4d3e4808d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:08.317487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866c6b1-e071-419c-8769-e4d3e4808d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:08.319430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d866c6b1-e071-419c-8769-e4d3e4808d76 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:08.437258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1411/2822] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:33:08 2022] POST /identity/v3/projects => generated 405 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:08.441388 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b66770-9097-4d34-b6b3-8d86e2ec4335 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:08.452253 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b66770-9097-4d34-b6b3-8d86e2ec4335 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:08.513143 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8515d0-bb5c-4bf9-8091-43ec3b345f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:08.513771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8515d0-bb5c-4bf9-8091-43ec3b345f97 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:08.514102 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8515d0-bb5c-4bf9-8091-43ec3b345f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:08.514376 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8515d0-bb5c-4bf9-8091-43ec3b345f97 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:08.515028 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8515d0-bb5c-4bf9-8091-43ec3b345f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:08.516961 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8515d0-bb5c-4bf9-8091-43ec3b345f97 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:08.584259 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1412/2823] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:33:08 2022] POST /identity/v3/users => generated 303 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:33:08.588565 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866c6b1-e071-419c-8769-e4d3e4808d76 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:08.598990 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d866c6b1-e071-419c-8769-e4d3e4808d76 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:08.604613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26a26b-4da3-4d57-978c-6fc2ac8e7c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:08.605211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26a26b-4da3-4d57-978c-6fc2ac8e7c2e admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:08.605484 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26a26b-4da3-4d57-978c-6fc2ac8e7c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:08.605749 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c26a26b-4da3-4d57-978c-6fc2ac8e7c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:08.606631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26a26b-4da3-4d57-978c-6fc2ac8e7c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:08.608234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c26a26b-4da3-4d57-978c-6fc2ac8e7c2e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:08.618429 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1412/2824] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:08.622526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8515d0-bb5c-4bf9-8091-43ec3b345f97 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:08.636062 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8515d0-bb5c-4bf9-8091-43ec3b345f97 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:08.643604 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fdef36-548e-4544-8742-7d4c38bdd683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:08.644542 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdef36-548e-4544-8742-7d4c38bdd683 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:08.645033 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdef36-548e-4544-8742-7d4c38bdd683 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:08.645463 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fdef36-548e-4544-8742-7d4c38bdd683 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:08.646554 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fdef36-548e-4544-8742-7d4c38bdd683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:08.648862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fdef36-548e-4544-8742-7d4c38bdd683 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:08.666280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1413/2825] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:08.670561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26a26b-4da3-4d57-978c-6fc2ac8e7c2e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:08.680850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c26a26b-4da3-4d57-978c-6fc2ac8e7c2e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:08.686539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f5fb5-6704-46e8-8849-9a7fa6b3851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:08.687173 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f5fb5-6704-46e8-8849-9a7fa6b3851f admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:08.687538 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f5fb5-6704-46e8-8849-9a7fa6b3851f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:08.687739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f5fb5-6704-46e8-8849-9a7fa6b3851f admin admin] PATH_INFO: `/v3/projects/8dce7868d1b24c4995be9e7c46f46814/users/f59eda4dc5754613a4c910cbcae2b613/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:08.708750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f5fb5-6704-46e8-8849-9a7fa6b3851f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dce7868d1b24c4995be9e7c46f46814, user_id=f59eda4dc5754613a4c910cbcae2b613, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:08.711472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f5fb5-6704-46e8-8849-9a7fa6b3851f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:08.779746 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:08.784542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1413/2826] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:08 2022] PUT /identity/v3/projects/8dce7868d1b24c4995be9e7c46f46814/users/f59eda4dc5754613a4c910cbcae2b613/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:08.802524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449bd80-2220-4dd5-a394-7fe2f5c78082 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:08.802797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449bd80-2220-4dd5-a394-7fe2f5c78082 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:08.803063 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449bd80-2220-4dd5-a394-7fe2f5c78082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:08.831936 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f449bd80-2220-4dd5-a394-7fe2f5c78082 None None] MFA Rules not processed for user `f59eda4dc5754613a4c910cbcae2b613`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:08.833144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f449bd80-2220-4dd5-a394-7fe2f5c78082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:08.901790 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1414/2827] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:08 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:08.906411 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f5fb5-6704-46e8-8849-9a7fa6b3851f admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:08.967416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f5fb5-6704-46e8-8849-9a7fa6b3851f admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:08.972264 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a24a8-0a53-4f96-a2d5-73658e3b99b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:08.972859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a24a8-0a53-4f96-a2d5-73658e3b99b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:08.973134 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a24a8-0a53-4f96-a2d5-73658e3b99b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:08.973401 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a24a8-0a53-4f96-a2d5-73658e3b99b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:08.974056 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a24a8-0a53-4f96-a2d5-73658e3b99b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:08.975946 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a24a8-0a53-4f96-a2d5-73658e3b99b9 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:09.077524 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1414/2828] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:33:08 2022] POST /identity/v3/projects => generated 403 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:09.081750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449bd80-2220-4dd5-a394-7fe2f5c78082 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:09.093429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449bd80-2220-4dd5-a394-7fe2f5c78082 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:09.101854 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:09.151386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187f355-0648-4e27-a580-42c1da23d365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:09.151986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f355-0648-4e27-a580-42c1da23d365 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:09.152257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f355-0648-4e27-a580-42c1da23d365 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:09.152523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f355-0648-4e27-a580-42c1da23d365 admin admin] PATH_INFO: `/v3/users` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:09.153183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187f355-0648-4e27-a580-42c1da23d365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:09.155428 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187f355-0648-4e27-a580-42c1da23d365 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:09.258598 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1415/2829] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:33:09 2022] POST /identity/v3/users => generated 308 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:09.263516 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a24a8-0a53-4f96-a2d5-73658e3b99b9 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:09.274267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a24a8-0a53-4f96-a2d5-73658e3b99b9 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:09.279531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4324e-2c42-4487-aba5-7a5958b6feaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:09.280123 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4324e-2c42-4487-aba5-7a5958b6feaf admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:09.280403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4324e-2c42-4487-aba5-7a5958b6feaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:09.280675 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4324e-2c42-4487-aba5-7a5958b6feaf admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:09.281345 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4324e-2c42-4487-aba5-7a5958b6feaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:09.282984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4324e-2c42-4487-aba5-7a5958b6feaf admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:09.293730 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1415/2830] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:09.297897 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187f355-0648-4e27-a580-42c1da23d365 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:09.308686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187f355-0648-4e27-a580-42c1da23d365 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:09.313730 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ded1c6-3723-4f15-8a97-4cf3afacf4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:09.314434 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ded1c6-3723-4f15-8a97-4cf3afacf4a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:09.314725 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ded1c6-3723-4f15-8a97-4cf3afacf4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:09.315027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ded1c6-3723-4f15-8a97-4cf3afacf4a2 admin admin] PATH_INFO: `/v3/projects/f2e449bc814a4a9593458549d5f54d66/users/5f77712ee5c749c69cda6745bd2545f8/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:09.331501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ded1c6-3723-4f15-8a97-4cf3afacf4a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e449bc814a4a9593458549d5f54d66, user_id=5f77712ee5c749c69cda6745bd2545f8, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:09.334292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ded1c6-3723-4f15-8a97-4cf3afacf4a2 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:09.419346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1416/2831] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:09 2022] PUT /identity/v3/projects/f2e449bc814a4a9593458549d5f54d66/users/5f77712ee5c749c69cda6745bd2545f8/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:09.423799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4324e-2c42-4487-aba5-7a5958b6feaf admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:09.434705 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4324e-2c42-4487-aba5-7a5958b6feaf admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:09.490510 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6c36f-8ad0-47b4-8394-f5903db39457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:09.491120 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6c36f-8ad0-47b4-8394-f5903db39457 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:09.491546 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6c36f-8ad0-47b4-8394-f5903db39457 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:09.491835 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6c36f-8ad0-47b4-8394-f5903db39457 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:09.492489 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6c36f-8ad0-47b4-8394-f5903db39457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:09.494704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6c36f-8ad0-47b4-8394-f5903db39457 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:09.506067 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1416/2832] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:33:09.510437 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ded1c6-3723-4f15-8a97-4cf3afacf4a2 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:09.521269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ded1c6-3723-4f15-8a97-4cf3afacf4a2 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:09.526023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c38d67-9e7a-4604-8b59-2b4db43485b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:09.526763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c38d67-9e7a-4604-8b59-2b4db43485b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:09.526948 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c38d67-9e7a-4604-8b59-2b4db43485b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:09.527219 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c38d67-9e7a-4604-8b59-2b4db43485b2 admin admin] PATH_INFO: `/v3/projects/f2e449bc814a4a9593458549d5f54d66/users/5f77712ee5c749c69cda6745bd2545f8/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:09.529513 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c38d67-9e7a-4604-8b59-2b4db43485b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e449bc814a4a9593458549d5f54d66, user_id=5f77712ee5c749c69cda6745bd2545f8, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:09.532793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c38d67-9e7a-4604-8b59-2b4db43485b2 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:09.543627 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1417/2833] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:09 2022] PUT /identity/v3/projects/f2e449bc814a4a9593458549d5f54d66/users/5f77712ee5c749c69cda6745bd2545f8/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:33:09.555107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad6756-30eb-47e8-8e06-361c86f75d43 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:09.556050 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad6756-30eb-47e8-8e06-361c86f75d43 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:09.556330 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad6756-30eb-47e8-8e06-361c86f75d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:09.584337 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-0aad6756-30eb-47e8-8e06-361c86f75d43 None None] MFA Rules not processed for user `5f77712ee5c749c69cda6745bd2545f8`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:09.585562 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-0aad6756-30eb-47e8-8e06-361c86f75d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:09.645727 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:09.654598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1417/2834] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:09 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:09.665751 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c38d67-9e7a-4604-8b59-2b4db43485b2 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:09.667505 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-a8c38d67-9e7a-4604-8b59-2b4db43485b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:09.726745 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c38d67-9e7a-4604-8b59-2b4db43485b2 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:09.731193 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d06db7-ceb2-4a8c-95f1-3d2630751dbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:09.731787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d06db7-ceb2-4a8c-95f1-3d2630751dbf service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:09.732443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d06db7-ceb2-4a8c-95f1-3d2630751dbf service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:09.732443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d06db7-ceb2-4a8c-95f1-3d2630751dbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:09.741558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d06db7-ceb2-4a8c-95f1-3d2630751dbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:09.743626 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d06db7-ceb2-4a8c-95f1-3d2630751dbf service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:09.753232 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1418/2835] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:09 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:09.889438 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad6756-30eb-47e8-8e06-361c86f75d43 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:09.950430 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad6756-30eb-47e8-8e06-361c86f75d43 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:09.955320 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:09.955987 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:09.956265 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:09.956527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] PATH_INFO: `/v3/users/f59eda4dc5754613a4c910cbcae2b613` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:09.957678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f59eda4dc5754613a4c910cbcae2b613)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:09.959715 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:10.162992 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f59eda4dc5754613a4c910cbcae2b613'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:10.257613 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f59eda4dc5754613a4c910cbcae2b613'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:10.266036 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f59eda4dc5754613a4c910cbcae2b613'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:10.279427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f59eda4dc5754613a4c910cbcae2b613'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:10.280163 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f59eda4dc5754613a4c910cbcae2b613'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:10.286710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1418/2836] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:33:09 2022] DELETE /identity/v3/users/f59eda4dc5754613a4c910cbcae2b613 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:10.291506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ae3d1-47ac-4bd4-8130-a943aeac6503 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:10.291813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ae3d1-47ac-4bd4-8130-a943aeac6503 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:10.292077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0ae3d1-47ac-4bd4-8130-a943aeac6503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:10.310838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-8e0ae3d1-47ac-4bd4-8130-a943aeac6503 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:10.312022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8e0ae3d1-47ac-4bd4-8130-a943aeac6503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:10.371511 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1419/2837] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:10.380896 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:10.382421 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:10.440786 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5ba18-9614-4b11-a649-975f51d97f36 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:10.444375 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988fbd3-19ea-4740-9350-2c2b4f7f34aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:10.445015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988fbd3-19ea-4740-9350-2c2b4f7f34aa service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:10.445286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988fbd3-19ea-4740-9350-2c2b4f7f34aa service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:10.445557 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988fbd3-19ea-4740-9350-2c2b4f7f34aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:10.455420 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988fbd3-19ea-4740-9350-2c2b4f7f34aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:10.456845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988fbd3-19ea-4740-9350-2c2b4f7f34aa service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:10.466841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1419/2838] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:10.888538 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ae3d1-47ac-4bd4-8130-a943aeac6503 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:10.890059 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-8e0ae3d1-47ac-4bd4-8130-a943aeac6503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:10.902350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0ae3d1-47ac-4bd4-8130-a943aeac6503 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:10.907348 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:10.908027 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:10.908309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:10.908578 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] PATH_INFO: `/v3/projects/8dce7868d1b24c4995be9e7c46f46814` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:10.909767 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dce7868d1b24c4995be9e7c46f46814)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:10.911755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:10.913521 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:11.029844 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dce7868d1b24c4995be9e7c46f46814', 'request_id': 'req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:11.137664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dce7868d1b24c4995be9e7c46f46814', 'request_id': 'req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:11.144458 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dce7868d1b24c4995be9e7c46f46814', 'request_id': 'req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:11.154813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] The token cache is being invalidate because project 8dce7868d1b24c4995be9e7c46f46814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:33:11.155045 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:11.155793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:11.157353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1420/2839] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:33:10 2022] DELETE /identity/v3/projects/8dce7868d1b24c4995be9e7c46f46814 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:11.161511 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988fbd3-19ea-4740-9350-2c2b4f7f34aa service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:11.171806 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988fbd3-19ea-4740-9350-2c2b4f7f34aa service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:11.184793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:11.231504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:11.232412 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:11.232687 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:11.232959 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] PATH_INFO: `/v3/users/5f77712ee5c749c69cda6745bd2545f8` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:11.234301 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f77712ee5c749c69cda6745bd2545f8)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:11.236223 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:11.426270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f77712ee5c749c69cda6745bd2545f8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:11.514984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f77712ee5c749c69cda6745bd2545f8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:11.522754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f77712ee5c749c69cda6745bd2545f8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:11.535796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f77712ee5c749c69cda6745bd2545f8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:11.536577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-f9acda89-4b1b-4373-9f9d-4c6f9bec70d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f77712ee5c749c69cda6745bd2545f8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:11.543131 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1420/2840] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:33:11 2022] DELETE /identity/v3/users/5f77712ee5c749c69cda6745bd2545f8 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:11.951254 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:11.952848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:11.968459 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700bcaa-1849-4c6e-9086-53cfbfeb0ef2 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:12.028192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:12.028847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:12.029127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:12.029396 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] PATH_INFO: `/v3/projects/f2e449bc814a4a9593458549d5f54d66` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:12.030592 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2e449bc814a4a9593458549d5f54d66)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:12.037676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:12.138373 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2e449bc814a4a9593458549d5f54d66', 'request_id': 'req-85e60d5b-5a28-416a-a823-c7c005f1f05b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:12.206690 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2e449bc814a4a9593458549d5f54d66', 'request_id': 'req-85e60d5b-5a28-416a-a823-c7c005f1f05b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:12.213899 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2e449bc814a4a9593458549d5f54d66', 'request_id': 'req-85e60d5b-5a28-416a-a823-c7c005f1f05b'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:12.224522 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] The token cache is being invalidate because project f2e449bc814a4a9593458549d5f54d66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:33:12.224800 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:12.225540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-85e60d5b-5a28-416a-a823-c7c005f1f05b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:12.227002 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1421/2841] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:33:11 2022] DELETE /identity/v3/projects/f2e449bc814a4a9593458549d5f54d66 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:12.232132 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f018df4-1f89-47a9-9925-3fff2e08eb04 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:12.232607 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f018df4-1f89-47a9-9925-3fff2e08eb04 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:12.238286 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f018df4-1f89-47a9-9925-3fff2e08eb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:12.253439 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8f018df4-1f89-47a9-9925-3fff2e08eb04 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:12.254631 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8f018df4-1f89-47a9-9925-3fff2e08eb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:12.313455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1421/2842] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:12.322060 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcecd0-1d57-4d01-9810-55cc48c509a3 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:12.322361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcecd0-1d57-4d01-9810-55cc48c509a3 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:12.322630 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcecd0-1d57-4d01-9810-55cc48c509a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:12.344512 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-4cfcecd0-1d57-4d01-9810-55cc48c509a3 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:12.345715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-4cfcecd0-1d57-4d01-9810-55cc48c509a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:12.353857 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1422/2843] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:12.358219 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f018df4-1f89-47a9-9925-3fff2e08eb04 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:12.368678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f018df4-1f89-47a9-9925-3fff2e08eb04 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:12.373315 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eef8dc-b5d5-41ee-af81-1b585fa4cb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:12.373907 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eef8dc-b5d5-41ee-af81-1b585fa4cb33 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:12.374207 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eef8dc-b5d5-41ee-af81-1b585fa4cb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:12.374474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eef8dc-b5d5-41ee-af81-1b585fa4cb33 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:12.375110 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eef8dc-b5d5-41ee-af81-1b585fa4cb33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:33:12.375407 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eef8dc-b5d5-41ee-af81-1b585fa4cb33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:12.377019 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eef8dc-b5d5-41ee-af81-1b585fa4cb33 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:12.389340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1422/2844] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:33:12.393815 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcecd0-1d57-4d01-9810-55cc48c509a3 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:12.402990 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfcecd0-1d57-4d01-9810-55cc48c509a3 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:12.407803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037dc9e-b445-468c-adff-1f4a6fda5a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:12.408414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037dc9e-b445-468c-adff-1f4a6fda5a73 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:12.410607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037dc9e-b445-468c-adff-1f4a6fda5a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:12.410607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037dc9e-b445-468c-adff-1f4a6fda5a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:12.410607 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037dc9e-b445-468c-adff-1f4a6fda5a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:12.411467 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037dc9e-b445-468c-adff-1f4a6fda5a73 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:12.423242 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:12.499818 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1423/2845] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:33:12 2022] POST /identity/v3/projects => generated 355 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:33:12.503927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eef8dc-b5d5-41ee-af81-1b585fa4cb33 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:12.513986 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eef8dc-b5d5-41ee-af81-1b585fa4cb33 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:12.534491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:12.570890 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f26b1d-b0d8-432e-bd17-a6beda6fa029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:12.571521 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f26b1d-b0d8-432e-bd17-a6beda6fa029 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:12.571796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f26b1d-b0d8-432e-bd17-a6beda6fa029 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:12.572064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f26b1d-b0d8-432e-bd17-a6beda6fa029 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:12.572716 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f26b1d-b0d8-432e-bd17-a6beda6fa029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:12.574638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f26b1d-b0d8-432e-bd17-a6beda6fa029 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:12.669658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1423/2846] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:33:12 2022] POST /identity/v3/users => generated 278 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:12.673805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037dc9e-b445-468c-adff-1f4a6fda5a73 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:12.685026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037dc9e-b445-468c-adff-1f4a6fda5a73 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:12.690054 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b463ec3-f1c3-4adb-95ff-634915af620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:12.690639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b463ec3-f1c3-4adb-95ff-634915af620b admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:12.690916 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b463ec3-f1c3-4adb-95ff-634915af620b admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:12.691184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b463ec3-f1c3-4adb-95ff-634915af620b admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:12.691982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b463ec3-f1c3-4adb-95ff-634915af620b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:12.693591 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b463ec3-f1c3-4adb-95ff-634915af620b admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:12.704030 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1424/2847] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:12.708070 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f26b1d-b0d8-432e-bd17-a6beda6fa029 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:12.718159 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f26b1d-b0d8-432e-bd17-a6beda6fa029 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:12.722833 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d9b5e-f8e2-4855-a21e-eda9d446e699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:12.723418 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d9b5e-f8e2-4855-a21e-eda9d446e699 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:12.723763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d9b5e-f8e2-4855-a21e-eda9d446e699 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:12.724053 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d9b5e-f8e2-4855-a21e-eda9d446e699 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:12.724708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d9b5e-f8e2-4855-a21e-eda9d446e699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:12.726311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d9b5e-f8e2-4855-a21e-eda9d446e699 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:12.736995 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1424/2848] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:12.740816 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b463ec3-f1c3-4adb-95ff-634915af620b admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:12.751450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b463ec3-f1c3-4adb-95ff-634915af620b admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:12.756205 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25614a-494e-45a8-b835-eebb04b00311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:12.756840 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25614a-494e-45a8-b835-eebb04b00311 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:12.757139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25614a-494e-45a8-b835-eebb04b00311 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:12.757406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25614a-494e-45a8-b835-eebb04b00311 admin admin] PATH_INFO: `/v3/projects/5053cef26be84d4bbd6531213eb7baf7/users/3d073169aef14c238a41f1fb897ecbad/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:12.771755 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25614a-494e-45a8-b835-eebb04b00311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5053cef26be84d4bbd6531213eb7baf7, user_id=3d073169aef14c238a41f1fb897ecbad, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:12.774392 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25614a-494e-45a8-b835-eebb04b00311 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:12.853502 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1425/2849] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:12 2022] PUT /identity/v3/projects/5053cef26be84d4bbd6531213eb7baf7/users/3d073169aef14c238a41f1fb897ecbad/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:33:12.864156 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40bbb4-74b7-4d7c-97d4-cbeadabac247 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:12.864437 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40bbb4-74b7-4d7c-97d4-cbeadabac247 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:12.864704 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40bbb4-74b7-4d7c-97d4-cbeadabac247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:12.892080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-eb40bbb4-74b7-4d7c-97d4-cbeadabac247 None None] MFA Rules not processed for user `3d073169aef14c238a41f1fb897ecbad`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:12.893284 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-eb40bbb4-74b7-4d7c-97d4-cbeadabac247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:12.957888 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1425/2850] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:12.962974 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25614a-494e-45a8-b835-eebb04b00311 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:13.024746 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25614a-494e-45a8-b835-eebb04b00311 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:13.029350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3070e48-a1d0-4b4a-840b-d8d7f479fa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:13.029942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3070e48-a1d0-4b4a-840b-d8d7f479fa00 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:13.030370 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3070e48-a1d0-4b4a-840b-d8d7f479fa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:13.030638 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3070e48-a1d0-4b4a-840b-d8d7f479fa00 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:13.031261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3070e48-a1d0-4b4a-840b-d8d7f479fa00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:13.033136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3070e48-a1d0-4b4a-840b-d8d7f479fa00 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:13.145121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1426/2851] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:33:12 2022] POST /identity/v3/projects => generated 357 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:13.149721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40bbb4-74b7-4d7c-97d4-cbeadabac247 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:13.160536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40bbb4-74b7-4d7c-97d4-cbeadabac247 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:13.217258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf2bd-857e-4eec-98fc-f1d3e6a1840f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:13.217978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf2bd-857e-4eec-98fc-f1d3e6a1840f admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:13.218266 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf2bd-857e-4eec-98fc-f1d3e6a1840f admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:13.218531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf2bd-857e-4eec-98fc-f1d3e6a1840f admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:13.219137 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cf2bd-857e-4eec-98fc-f1d3e6a1840f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:13.221060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8cf2bd-857e-4eec-98fc-f1d3e6a1840f admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:13.309874 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1426/2852] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:33:13 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:13.314148 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3070e48-a1d0-4b4a-840b-d8d7f479fa00 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:13.325896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3070e48-a1d0-4b4a-840b-d8d7f479fa00 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:13.330741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1fdc9-0f87-412b-8a35-444299d23448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:13.331343 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1fdc9-0f87-412b-8a35-444299d23448 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:13.331623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1fdc9-0f87-412b-8a35-444299d23448 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:13.331900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b1fdc9-0f87-412b-8a35-444299d23448 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:13.332559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1fdc9-0f87-412b-8a35-444299d23448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:13.334239 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b1fdc9-0f87-412b-8a35-444299d23448 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:13.345596 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1427/2853] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:13.350142 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf2bd-857e-4eec-98fc-f1d3e6a1840f admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:13.360083 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf2bd-857e-4eec-98fc-f1d3e6a1840f admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:13.365095 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f798f-6420-4722-a21b-9724af078dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:13.365770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f798f-6420-4722-a21b-9724af078dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:13.366064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f798f-6420-4722-a21b-9724af078dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:13.366354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f798f-6420-4722-a21b-9724af078dc2 admin admin] PATH_INFO: `/v3/projects/10f2941445b143dea770b32c9bc97d0c/users/5d4d3401b80c4675adcc1231cdbb70d9/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:13.383277 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f798f-6420-4722-a21b-9724af078dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f2941445b143dea770b32c9bc97d0c, user_id=5d4d3401b80c4675adcc1231cdbb70d9, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:13.386531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f798f-6420-4722-a21b-9724af078dc2 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:13.452912 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:13.458781 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1427/2854] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:13 2022] PUT /identity/v3/projects/10f2941445b143dea770b32c9bc97d0c/users/5d4d3401b80c4675adcc1231cdbb70d9/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:13.463444 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1fdc9-0f87-412b-8a35-444299d23448 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:13.473461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b1fdc9-0f87-412b-8a35-444299d23448 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:13.530469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaef56-56ce-430e-a019-3487d30ba1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:13.531083 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaef56-56ce-430e-a019-3487d30ba1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:13.531363 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaef56-56ce-430e-a019-3487d30ba1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:13.531629 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaef56-56ce-430e-a019-3487d30ba1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:13.532270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaef56-56ce-430e-a019-3487d30ba1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:13.533851 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaef56-56ce-430e-a019-3487d30ba1d1 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:13.544145 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1428/2855] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:33:13.548634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f798f-6420-4722-a21b-9724af078dc2 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:13.559741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f798f-6420-4722-a21b-9724af078dc2 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:13.564670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ef082-213c-4b82-b794-61ba0f3666dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:13.565240 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ef082-213c-4b82-b794-61ba0f3666dc admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:13.565519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ef082-213c-4b82-b794-61ba0f3666dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:13.565791 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ef082-213c-4b82-b794-61ba0f3666dc admin admin] PATH_INFO: `/v3/projects/10f2941445b143dea770b32c9bc97d0c/users/5d4d3401b80c4675adcc1231cdbb70d9/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:13.567733 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ef082-213c-4b82-b794-61ba0f3666dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f2941445b143dea770b32c9bc97d0c, user_id=5d4d3401b80c4675adcc1231cdbb70d9, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:13.570372 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61ef082-213c-4b82-b794-61ba0f3666dc admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:13.581628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1428/2856] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:13 2022] PUT /identity/v3/projects/10f2941445b143dea770b32c9bc97d0c/users/5d4d3401b80c4675adcc1231cdbb70d9/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:13.593104 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a9903-40f7-4d0c-8bc5-7b85e6f14e3d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:13.593399 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a9903-40f7-4d0c-8bc5-7b85e6f14e3d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:13.593669 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a9903-40f7-4d0c-8bc5-7b85e6f14e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:13.622377 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-769a9903-40f7-4d0c-8bc5-7b85e6f14e3d None None] MFA Rules not processed for user `5d4d3401b80c4675adcc1231cdbb70d9`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:13.623675 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-769a9903-40f7-4d0c-8bc5-7b85e6f14e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:13.637142 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:13.693122 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1429/2857] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:13 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:13.705271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ef082-213c-4b82-b794-61ba0f3666dc admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:13.706971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-e61ef082-213c-4b82-b794-61ba0f3666dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:13.764225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ef082-213c-4b82-b794-61ba0f3666dc admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:13.767814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9028efe-f5fb-4600-bfee-b1e9a0fc4c45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:13.768398 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9028efe-f5fb-4600-bfee-b1e9a0fc4c45 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:13.768672 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9028efe-f5fb-4600-bfee-b1e9a0fc4c45 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:13.768938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9028efe-f5fb-4600-bfee-b1e9a0fc4c45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:13.777942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9028efe-f5fb-4600-bfee-b1e9a0fc4c45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:13.779380 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9028efe-f5fb-4600-bfee-b1e9a0fc4c45 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:13.788628 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1429/2858] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:13 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:14.037769 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a9903-40f7-4d0c-8bc5-7b85e6f14e3d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:14.048587 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a9903-40f7-4d0c-8bc5-7b85e6f14e3d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:14.052255 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3998057-337a-4716-944b-d102005420a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:14.052852 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3998057-337a-4716-944b-d102005420a3 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:14.053128 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3998057-337a-4716-944b-d102005420a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:14.053395 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3998057-337a-4716-944b-d102005420a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:14.112096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3998057-337a-4716-944b-d102005420a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:14.113522 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3998057-337a-4716-944b-d102005420a3 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:14.123183 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1430/2859] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:23.784491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:24.118337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=50 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:33.085909 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9028efe-f5fb-4600-bfee-b1e9a0fc4c45 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:33.148213 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9028efe-f5fb-4600-bfee-b1e9a0fc4c45 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:33.153218 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:33.153827 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:33.154167 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:33.154532 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] PATH_INFO: `/v3/users/3d073169aef14c238a41f1fb897ecbad` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:33.155638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d073169aef14c238a41f1fb897ecbad)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:33.157531 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:33.379843 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d073169aef14c238a41f1fb897ecbad'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:33.452197 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d073169aef14c238a41f1fb897ecbad'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:33.459553 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d073169aef14c238a41f1fb897ecbad'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:33.473452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d073169aef14c238a41f1fb897ecbad'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:33.474175 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d073169aef14c238a41f1fb897ecbad'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:33.480280 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1430/2860] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:33:33 2022] DELETE /identity/v3/users/3d073169aef14c238a41f1fb897ecbad => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:33.484877 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182ee40-c45c-4315-954f-52b3f6a40f16 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:33.485929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182ee40-c45c-4315-954f-52b3f6a40f16 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:33.485929 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182ee40-c45c-4315-954f-52b3f6a40f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:33.504451 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-2182ee40-c45c-4315-954f-52b3f6a40f16 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:33.505715 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2182ee40-c45c-4315-954f-52b3f6a40f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:33.565516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1431/2861] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:33.575182 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:33.576904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:33.635839 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c734645-174c-4e14-9f47-d42c0f5ee3ef admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:33.639377 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7f94e-7497-4e11-9bb1-b7f9246fbde8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:33.639961 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7f94e-7497-4e11-9bb1-b7f9246fbde8 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:33.640236 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7f94e-7497-4e11-9bb1-b7f9246fbde8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:33.640505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7f94e-7497-4e11-9bb1-b7f9246fbde8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:33.650089 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7f94e-7497-4e11-9bb1-b7f9246fbde8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:33.651518 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7f94e-7497-4e11-9bb1-b7f9246fbde8 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:33.661543 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1431/2862] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:34.106553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182ee40-c45c-4315-954f-52b3f6a40f16 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:34.114281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-2182ee40-c45c-4315-954f-52b3f6a40f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:34.120722 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182ee40-c45c-4315-954f-52b3f6a40f16 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:34.125856 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:34.126515 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:34.126797 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:34.127068 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] PATH_INFO: `/v3/projects/5053cef26be84d4bbd6531213eb7baf7` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:34.128197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5053cef26be84d4bbd6531213eb7baf7)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:34.130167 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:34.277215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5053cef26be84d4bbd6531213eb7baf7', 'request_id': 'req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:34.350094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5053cef26be84d4bbd6531213eb7baf7', 'request_id': 'req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:34.356885 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5053cef26be84d4bbd6531213eb7baf7', 'request_id': 'req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:34.368228 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] The token cache is being invalidate because project 5053cef26be84d4bbd6531213eb7baf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:33:34.368534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:34.369172 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:34.370692 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1432/2863] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:33:34 2022] DELETE /identity/v3/projects/5053cef26be84d4bbd6531213eb7baf7 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:34.374805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7f94e-7497-4e11-9bb1-b7f9246fbde8 service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:34.385713 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7f94e-7497-4e11-9bb1-b7f9246fbde8 service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:34.403963 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:34.441340 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:34.442625 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:34.443140 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:34.443470 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] PATH_INFO: `/v3/users/5d4d3401b80c4675adcc1231cdbb70d9` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:34.444971 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d4d3401b80c4675adcc1231cdbb70d9)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:34.446948 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:34.654942 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d4d3401b80c4675adcc1231cdbb70d9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:34.739708 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d4d3401b80c4675adcc1231cdbb70d9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:34.746988 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d4d3401b80c4675adcc1231cdbb70d9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:34.758722 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d4d3401b80c4675adcc1231cdbb70d9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:34.759390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-37dbe39e-b2f3-4c2c-a48f-97309f59e042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d4d3401b80c4675adcc1231cdbb70d9'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:34.765187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1432/2864] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:33:34 2022] DELETE /identity/v3/users/5d4d3401b80c4675adcc1231cdbb70d9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:35.164044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:35.165506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:35.180347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b116b8e1-d1ab-4bcf-9020-ffb7bc46d167 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:35.237356 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:35.238018 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:35.238298 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:35.238639 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] PATH_INFO: `/v3/projects/10f2941445b143dea770b32c9bc97d0c` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:35.239793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f2941445b143dea770b32c9bc97d0c)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:35.241789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:35.329081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:35.345783 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f2941445b143dea770b32c9bc97d0c', 'request_id': 'req-b4988c64-e177-4965-b686-733b1d1c70a8'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:35.459870 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f2941445b143dea770b32c9bc97d0c', 'request_id': 'req-b4988c64-e177-4965-b686-733b1d1c70a8'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:35.466536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f2941445b143dea770b32c9bc97d0c', 'request_id': 'req-b4988c64-e177-4965-b686-733b1d1c70a8'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:35.476763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] The token cache is being invalidate because project 10f2941445b143dea770b32c9bc97d0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:33:35.477046 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:35.477762 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b4988c64-e177-4965-b686-733b1d1c70a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:35.479283 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1433/2865] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:33:35 2022] DELETE /identity/v3/projects/10f2941445b143dea770b32c9bc97d0c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:35.484153 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65156d41-2a12-4b14-9ed3-f979c859cc13 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:35.484630 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65156d41-2a12-4b14-9ed3-f979c859cc13 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:35.484742 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65156d41-2a12-4b14-9ed3-f979c859cc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:35.503903 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-65156d41-2a12-4b14-9ed3-f979c859cc13 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:35.505591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-65156d41-2a12-4b14-9ed3-f979c859cc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:35.564996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1433/2866] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:35.573238 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e8040-161d-4887-8b2c-7b621eeaef6e None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:35.573516 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e8040-161d-4887-8b2c-7b621eeaef6e None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:35.573780 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e8040-161d-4887-8b2c-7b621eeaef6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:35.593387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-470e8040-161d-4887-8b2c-7b621eeaef6e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:35.594686 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-470e8040-161d-4887-8b2c-7b621eeaef6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:35.602553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1434/2867] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:35.606744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65156d41-2a12-4b14-9ed3-f979c859cc13 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:35.616500 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65156d41-2a12-4b14-9ed3-f979c859cc13 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:35.621211 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cc268-75aa-4f12-91f0-7642d9054bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:35.621665 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cc268-75aa-4f12-91f0-7642d9054bdf admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:35.621938 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cc268-75aa-4f12-91f0-7642d9054bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:35.622231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045cc268-75aa-4f12-91f0-7642d9054bdf admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:35.622858 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cc268-75aa-4f12-91f0-7642d9054bdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:33:35.623154 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cc268-75aa-4f12-91f0-7642d9054bdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:35.624707 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045cc268-75aa-4f12-91f0-7642d9054bdf admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:35.636483 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1434/2868] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:33:35.641156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e8040-161d-4887-8b2c-7b621eeaef6e None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:35.650949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e8040-161d-4887-8b2c-7b621eeaef6e None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:35.656137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc538e-344d-41a8-9284-6c0fb99332c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:35.656719 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc538e-344d-41a8-9284-6c0fb99332c7 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:35.656993 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc538e-344d-41a8-9284-6c0fb99332c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:35.657268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc538e-344d-41a8-9284-6c0fb99332c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:35.657920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc538e-344d-41a8-9284-6c0fb99332c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:35.662693 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc538e-344d-41a8-9284-6c0fb99332c7 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:35.775384 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1435/2869] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:33:35 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 09 17:33:35.779957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cc268-75aa-4f12-91f0-7642d9054bdf admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:35.790020 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045cc268-75aa-4f12-91f0-7642d9054bdf admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:35.815423 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:35.845845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c9753-cbdd-48cc-b75f-8999e9659845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:35.846465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c9753-cbdd-48cc-b75f-8999e9659845 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:35.846741 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c9753-cbdd-48cc-b75f-8999e9659845 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:35.847015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c9753-cbdd-48cc-b75f-8999e9659845 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:35.847635 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c9753-cbdd-48cc-b75f-8999e9659845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:35.849528 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9c9753-cbdd-48cc-b75f-8999e9659845 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:35.939225 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1435/2870] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:33:35 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:35.944113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc538e-344d-41a8-9284-6c0fb99332c7 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:35.954824 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc538e-344d-41a8-9284-6c0fb99332c7 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:35.960614 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500d631-50c3-40ab-a31c-18d8422d61d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:35.961202 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500d631-50c3-40ab-a31c-18d8422d61d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:35.961481 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500d631-50c3-40ab-a31c-18d8422d61d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:35.961753 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500d631-50c3-40ab-a31c-18d8422d61d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:35.962429 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500d631-50c3-40ab-a31c-18d8422d61d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:35.963996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500d631-50c3-40ab-a31c-18d8422d61d3 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:35.974208 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1436/2871] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:35.978440 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c9753-cbdd-48cc-b75f-8999e9659845 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:35.989152 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c9753-cbdd-48cc-b75f-8999e9659845 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:35.994008 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d9ca0-c6d1-489b-ad2d-a3da9e13f134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:35.994920 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d9ca0-c6d1-489b-ad2d-a3da9e13f134 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:35.995204 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d9ca0-c6d1-489b-ad2d-a3da9e13f134 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:35.995479 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d9ca0-c6d1-489b-ad2d-a3da9e13f134 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:35.996107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d9ca0-c6d1-489b-ad2d-a3da9e13f134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:35.997744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d9ca0-c6d1-489b-ad2d-a3da9e13f134 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:36.008870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1436/2872] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:36.013123 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500d631-50c3-40ab-a31c-18d8422d61d3 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:36.023362 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500d631-50c3-40ab-a31c-18d8422d61d3 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:36.028147 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adbc95-40fa-4310-9674-a70396ff8d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:36.028763 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adbc95-40fa-4310-9674-a70396ff8d2f admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:36.029039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adbc95-40fa-4310-9674-a70396ff8d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:36.029315 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0adbc95-40fa-4310-9674-a70396ff8d2f admin admin] PATH_INFO: `/v3/projects/6cb04af291394217ae60bfd42f15188b/users/1cbfea6f7f184f0eb88cded90944a9b8/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:36.044992 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adbc95-40fa-4310-9674-a70396ff8d2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb04af291394217ae60bfd42f15188b, user_id=1cbfea6f7f184f0eb88cded90944a9b8, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:36.047661 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0adbc95-40fa-4310-9674-a70396ff8d2f admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:36.116863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1437/2873] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:36 2022] PUT /identity/v3/projects/6cb04af291394217ae60bfd42f15188b/users/1cbfea6f7f184f0eb88cded90944a9b8/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:36.128550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f5a9f-a4cb-4be4-9463-c94c5fe87e6a None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:36.128838 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f5a9f-a4cb-4be4-9463-c94c5fe87e6a None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:36.129109 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f5a9f-a4cb-4be4-9463-c94c5fe87e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:36.163115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-5c3f5a9f-a4cb-4be4-9463-c94c5fe87e6a None None] MFA Rules not processed for user `1cbfea6f7f184f0eb88cded90944a9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:36.164355 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5c3f5a9f-a4cb-4be4-9463-c94c5fe87e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:36.228763 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1437/2874] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:36.233350 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adbc95-40fa-4310-9674-a70396ff8d2f admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:36.297921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0adbc95-40fa-4310-9674-a70396ff8d2f admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:36.303234 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7e60f-7e9f-46b0-8f9e-4ddb6ca9fc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:36.303921 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7e60f-7e9f-46b0-8f9e-4ddb6ca9fc36 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:36.304330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7e60f-7e9f-46b0-8f9e-4ddb6ca9fc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:36.304696 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7e60f-7e9f-46b0-8f9e-4ddb6ca9fc36 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:36.305402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7e60f-7e9f-46b0-8f9e-4ddb6ca9fc36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:36.307406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7e60f-7e9f-46b0-8f9e-4ddb6ca9fc36 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:36.433807 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1438/2875] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:33:36 2022] POST /identity/v3/projects => generated 369 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:36.437386 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f5a9f-a4cb-4be4-9463-c94c5fe87e6a None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:36.450133 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f5a9f-a4cb-4be4-9463-c94c5fe87e6a None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:36.513308 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dc611-4f80-4f09-a5e5-f894fe904cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:36.513934 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dc611-4f80-4f09-a5e5-f894fe904cff admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:36.514379 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dc611-4f80-4f09-a5e5-f894fe904cff admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:36.514645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5dc611-4f80-4f09-a5e5-f894fe904cff admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:36.515276 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5dc611-4f80-4f09-a5e5-f894fe904cff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:36.517231 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5dc611-4f80-4f09-a5e5-f894fe904cff admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:36.630689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1438/2876] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:33:36 2022] POST /identity/v3/users => generated 291 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:36.634685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7e60f-7e9f-46b0-8f9e-4ddb6ca9fc36 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:36.645524 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7e60f-7e9f-46b0-8f9e-4ddb6ca9fc36 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:36.650260 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b61bb-f4f3-46ab-9886-ca1ed227d650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:36.650868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b61bb-f4f3-46ab-9886-ca1ed227d650 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:36.651139 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b61bb-f4f3-46ab-9886-ca1ed227d650 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:36.651402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b61bb-f4f3-46ab-9886-ca1ed227d650 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:36.652081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b61bb-f4f3-46ab-9886-ca1ed227d650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:36.653637 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b61bb-f4f3-46ab-9886-ca1ed227d650 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:36.664771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1439/2877] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:36.669119 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dc611-4f80-4f09-a5e5-f894fe904cff admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:36.679578 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5dc611-4f80-4f09-a5e5-f894fe904cff admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:36.684144 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd94d1-eeab-45c7-b127-9aa8c20cf427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:36.687405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd94d1-eeab-45c7-b127-9aa8c20cf427 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:36.687405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd94d1-eeab-45c7-b127-9aa8c20cf427 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:36.687405 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd94d1-eeab-45c7-b127-9aa8c20cf427 admin admin] PATH_INFO: `/v3/projects/b7367da3e561453dbe73d0778eeffc7a/users/ae0815fa24904ee99070e5dbd13fd677/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:36.700107 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd94d1-eeab-45c7-b127-9aa8c20cf427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7367da3e561453dbe73d0778eeffc7a, user_id=ae0815fa24904ee99070e5dbd13fd677, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:36.702859 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd94d1-eeab-45c7-b127-9aa8c20cf427 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:36.785138 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1439/2878] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:36 2022] PUT /identity/v3/projects/b7367da3e561453dbe73d0778eeffc7a/users/ae0815fa24904ee99070e5dbd13fd677/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:36.789275 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b61bb-f4f3-46ab-9886-ca1ed227d650 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:36.799862 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b61bb-f4f3-46ab-9886-ca1ed227d650 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:36.820736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:36.857585 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52db61b-a003-456b-9547-c6e0aef6e209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:36.858535 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52db61b-a003-456b-9547-c6e0aef6e209 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:36.859057 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52db61b-a003-456b-9547-c6e0aef6e209 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:36.859853 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52db61b-a003-456b-9547-c6e0aef6e209 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:36.860613 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52db61b-a003-456b-9547-c6e0aef6e209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:36.862257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52db61b-a003-456b-9547-c6e0aef6e209 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:36.872612 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1440/2879] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:36.876939 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd94d1-eeab-45c7-b127-9aa8c20cf427 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:36.885789 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:36.891275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd94d1-eeab-45c7-b127-9aa8c20cf427 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:36.896677 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b0ff5-20b4-40ab-aafe-830587d7e140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:36.897394 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b0ff5-20b4-40ab-aafe-830587d7e140 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:36.897670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b0ff5-20b4-40ab-aafe-830587d7e140 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:36.897935 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b0ff5-20b4-40ab-aafe-830587d7e140 admin admin] PATH_INFO: `/v3/projects/b7367da3e561453dbe73d0778eeffc7a/users/ae0815fa24904ee99070e5dbd13fd677/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:36.902174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b0ff5-20b4-40ab-aafe-830587d7e140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7367da3e561453dbe73d0778eeffc7a, user_id=ae0815fa24904ee99070e5dbd13fd677, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:36.902577 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b0ff5-20b4-40ab-aafe-830587d7e140 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:36.913064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1440/2880] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:36 2022] PUT /identity/v3/projects/b7367da3e561453dbe73d0778eeffc7a/users/ae0815fa24904ee99070e5dbd13fd677/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:36.923450 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9c14b-4056-43af-87c4-921a8d1933ad None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:36.923758 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9c14b-4056-43af-87c4-921a8d1933ad None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:36.924022 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9c14b-4056-43af-87c4-921a8d1933ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:36.953900 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-9ae9c14b-4056-43af-87c4-921a8d1933ad None None] MFA Rules not processed for user `ae0815fa24904ee99070e5dbd13fd677`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:36.955094 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9ae9c14b-4056-43af-87c4-921a8d1933ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:37.022540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1441/2881] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:37.033834 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b0ff5-20b4-40ab-aafe-830587d7e140 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:37.036076 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-302b0ff5-20b4-40ab-aafe-830587d7e140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:37.095724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b0ff5-20b4-40ab-aafe-830587d7e140 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:37.099351 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e56c53-8a9e-4758-80c0-91ea118ddd20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:37.099924 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e56c53-8a9e-4758-80c0-91ea118ddd20 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:37.100194 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e56c53-8a9e-4758-80c0-91ea118ddd20 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:37.100504 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e56c53-8a9e-4758-80c0-91ea118ddd20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:37.109744 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e56c53-8a9e-4758-80c0-91ea118ddd20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:37.111271 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e56c53-8a9e-4758-80c0-91ea118ddd20 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:37.123513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1441/2882] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:37.276023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9c14b-4056-43af-87c4-921a8d1933ad None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:37.277552 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-9ae9c14b-4056-43af-87c4-921a8d1933ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:37.289750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9c14b-4056-43af-87c4-921a8d1933ad None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:37.293287 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827fa41-0431-47b4-b27f-b7421a7af1a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:37.293863 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827fa41-0431-47b4-b27f-b7421a7af1a0 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:37.294168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827fa41-0431-47b4-b27f-b7421a7af1a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:37.294435 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827fa41-0431-47b4-b27f-b7421a7af1a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:37.352813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827fa41-0431-47b4-b27f-b7421a7af1a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:37.354236 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827fa41-0431-47b4-b27f-b7421a7af1a0 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:37.363180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1442/2883] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:42.113805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e56c53-8a9e-4758-80c0-91ea118ddd20 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:42.174643 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e56c53-8a9e-4758-80c0-91ea118ddd20 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:42.179750 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:42.181196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:42.181196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:42.181196 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] PATH_INFO: `/v3/users/1cbfea6f7f184f0eb88cded90944a9b8` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:42.182113 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cbfea6f7f184f0eb88cded90944a9b8)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:42.184084 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:42.365981 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cbfea6f7f184f0eb88cded90944a9b8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:42.424390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cbfea6f7f184f0eb88cded90944a9b8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:42.432762 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cbfea6f7f184f0eb88cded90944a9b8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:42.444978 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cbfea6f7f184f0eb88cded90944a9b8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:42.445629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cbfea6f7f184f0eb88cded90944a9b8'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:42.451638 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1442/2884] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:33:42 2022] DELETE /identity/v3/users/1cbfea6f7f184f0eb88cded90944a9b8 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:42.456285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42127d6d-b5f2-43a5-894f-fb096a750ce2 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:42.456788 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42127d6d-b5f2-43a5-894f-fb096a750ce2 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:42.456905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42127d6d-b5f2-43a5-894f-fb096a750ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:42.476180 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-42127d6d-b5f2-43a5-894f-fb096a750ce2 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:42.477414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-42127d6d-b5f2-43a5-894f-fb096a750ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:42.518896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=100 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:42.538741 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1443/2885] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:42.548431 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:42.549913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:42.564664 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:42.611177 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989260f-d649-41a6-b0c8-48738afe6d8c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:42.615096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67531ca9-f7ce-4e66-88fa-7d7e30409f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:42.615703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67531ca9-f7ce-4e66-88fa-7d7e30409f2c service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:42.616038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67531ca9-f7ce-4e66-88fa-7d7e30409f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:42.616251 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67531ca9-f7ce-4e66-88fa-7d7e30409f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:42.626572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67531ca9-f7ce-4e66-88fa-7d7e30409f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:42.627994 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67531ca9-f7ce-4e66-88fa-7d7e30409f2c service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:42.637550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1443/2886] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:43.073596 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42127d6d-b5f2-43a5-894f-fb096a750ce2 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:43.075081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-42127d6d-b5f2-43a5-894f-fb096a750ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:43.089213 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42127d6d-b5f2-43a5-894f-fb096a750ce2 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:43.093944 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:43.094594 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:43.094869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:43.095137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] PATH_INFO: `/v3/projects/6cb04af291394217ae60bfd42f15188b` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:43.096261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cb04af291394217ae60bfd42f15188b)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:43.098197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:43.198154 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cb04af291394217ae60bfd42f15188b', 'request_id': 'req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:43.299695 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cb04af291394217ae60bfd42f15188b', 'request_id': 'req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:43.307330 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cb04af291394217ae60bfd42f15188b', 'request_id': 'req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:43.318880 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] The token cache is being invalidate because project 6cb04af291394217ae60bfd42f15188b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:33:43.319161 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:43.319798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:43.321326 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1444/2887] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:33:43 2022] DELETE /identity/v3/projects/6cb04af291394217ae60bfd42f15188b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:43.325254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67531ca9-f7ce-4e66-88fa-7d7e30409f2c service neutron] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:43.335151 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67531ca9-f7ce-4e66-88fa-7d7e30409f2c service neutron] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:43.390573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:43.391183 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:43.391652 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:43.391796 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] PATH_INFO: `/v3/users/ae0815fa24904ee99070e5dbd13fd677` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:43.392830 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0815fa24904ee99070e5dbd13fd677)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:43.394754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:43.571116 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0815fa24904ee99070e5dbd13fd677'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:43.631186 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0815fa24904ee99070e5dbd13fd677'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:43.638710 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0815fa24904ee99070e5dbd13fd677'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:43.651272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0815fa24904ee99070e5dbd13fd677'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:43.652026 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-0b344ccd-6154-4e91-a4c1-ef8a47aa6c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0815fa24904ee99070e5dbd13fd677'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:43.658363 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1444/2888] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:33:43 2022] DELETE /identity/v3/users/ae0815fa24904ee99070e5dbd13fd677 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:44.101674 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:44.103152 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:44.116848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d15c3d-b5bb-4bc9-ac21-099a2d4ee7f0 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:44.173414 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:44.174061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:44.174335 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:44.174601 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] PATH_INFO: `/v3/projects/b7367da3e561453dbe73d0778eeffc7a` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:44.175691 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7367da3e561453dbe73d0778eeffc7a)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:44.177541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:44.305926 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7367da3e561453dbe73d0778eeffc7a', 'request_id': 'req-101c5408-3859-450e-8f8b-ea48fe419813'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:44.379588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7367da3e561453dbe73d0778eeffc7a', 'request_id': 'req-101c5408-3859-450e-8f8b-ea48fe419813'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:44.386986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7367da3e561453dbe73d0778eeffc7a', 'request_id': 'req-101c5408-3859-450e-8f8b-ea48fe419813'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:44.393523 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:44.397982 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] The token cache is being invalidate because project b7367da3e561453dbe73d0778eeffc7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:33:44.398269 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:44.398915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-101c5408-3859-450e-8f8b-ea48fe419813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:33:44.400394 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1445/2889] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:33:44 2022] DELETE /identity/v3/projects/b7367da3e561453dbe73d0778eeffc7a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:44.405451 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86d8c0-193a-49cc-a583-d6a4824e2500 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:44.405913 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86d8c0-193a-49cc-a583-d6a4824e2500 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:44.409724 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86d8c0-193a-49cc-a583-d6a4824e2500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:44.426121 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-ba86d8c0-193a-49cc-a583-d6a4824e2500 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:44.427316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-ba86d8c0-193a-49cc-a583-d6a4824e2500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:44.456540 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:44.490580 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1445/2890] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:44.499915 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e18dc-a0f2-4b0e-a190-d2103a7a22af None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:44.500224 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e18dc-a0f2-4b0e-a190-d2103a7a22af None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:44.500517 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e18dc-a0f2-4b0e-a190-d2103a7a22af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:44.523677 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b19e18dc-a0f2-4b0e-a190-d2103a7a22af None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:44.526076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b19e18dc-a0f2-4b0e-a190-d2103a7a22af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:44.535600 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1446/2891] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:44.540444 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86d8c0-193a-49cc-a583-d6a4824e2500 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:44.551065 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86d8c0-193a-49cc-a583-d6a4824e2500 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:44.556295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d79748-b5b8-4ed4-9387-d75c497dd627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:44.556875 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d79748-b5b8-4ed4-9387-d75c497dd627 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:44.557146 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d79748-b5b8-4ed4-9387-d75c497dd627 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:44.557410 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d79748-b5b8-4ed4-9387-d75c497dd627 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:44.558082 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d79748-b5b8-4ed4-9387-d75c497dd627 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:33:44.558403 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d79748-b5b8-4ed4-9387-d75c497dd627 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:44.559982 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d79748-b5b8-4ed4-9387-d75c497dd627 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:44.571536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1446/2892] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:33:44.575799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e18dc-a0f2-4b0e-a190-d2103a7a22af None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:44.585999 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e18dc-a0f2-4b0e-a190-d2103a7a22af None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:44.590770 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4baf43-1f54-4d76-b2fc-94981de724cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:44.591359 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4baf43-1f54-4d76-b2fc-94981de724cb admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:44.591640 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4baf43-1f54-4d76-b2fc-94981de724cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:44.591904 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4baf43-1f54-4d76-b2fc-94981de724cb admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:44.592520 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4baf43-1f54-4d76-b2fc-94981de724cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:44.594443 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4baf43-1f54-4d76-b2fc-94981de724cb admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:44.696263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1447/2893] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:33:44 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:44.700508 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d79748-b5b8-4ed4-9387-d75c497dd627 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:44.710940 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d79748-b5b8-4ed4-9387-d75c497dd627 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:44.767699 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e62d4-78f5-4ab6-8786-52deb7a45a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:44.768295 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e62d4-78f5-4ab6-8786-52deb7a45a4e admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:44.768573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e62d4-78f5-4ab6-8786-52deb7a45a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:44.768845 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e62d4-78f5-4ab6-8786-52deb7a45a4e admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:44.769456 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e62d4-78f5-4ab6-8786-52deb7a45a4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:44.771390 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e62d4-78f5-4ab6-8786-52deb7a45a4e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:44.854539 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1447/2894] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:33:44 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:44.858866 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4baf43-1f54-4d76-b2fc-94981de724cb admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:44.869431 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4baf43-1f54-4d76-b2fc-94981de724cb admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:44.874032 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a6712-0910-49a2-9a20-8a2cf4cb7f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:44.880644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a6712-0910-49a2-9a20-8a2cf4cb7f01 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:44.880644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a6712-0910-49a2-9a20-8a2cf4cb7f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:44.880644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a6712-0910-49a2-9a20-8a2cf4cb7f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:44.880644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a6712-0910-49a2-9a20-8a2cf4cb7f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:44.880644 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a6712-0910-49a2-9a20-8a2cf4cb7f01 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:44.887972 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1448/2895] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:33:44.892283 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e62d4-78f5-4ab6-8786-52deb7a45a4e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:44.903023 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e62d4-78f5-4ab6-8786-52deb7a45a4e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:44.907678 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab55e2-171c-4efb-9ef9-04fdd0f2aba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:44.908267 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab55e2-171c-4efb-9ef9-04fdd0f2aba3 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:44.908542 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab55e2-171c-4efb-9ef9-04fdd0f2aba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:44.908814 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab55e2-171c-4efb-9ef9-04fdd0f2aba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:44.909468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab55e2-171c-4efb-9ef9-04fdd0f2aba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:44.911078 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab55e2-171c-4efb-9ef9-04fdd0f2aba3 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:44.922881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1448/2896] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:44.927243 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a6712-0910-49a2-9a20-8a2cf4cb7f01 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:44.937293 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a6712-0910-49a2-9a20-8a2cf4cb7f01 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:44.942178 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a761c1-4c31-4ba3-b12f-dd8f648f446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:44.942811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a761c1-4c31-4ba3-b12f-dd8f648f446c admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:44.943087 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a761c1-4c31-4ba3-b12f-dd8f648f446c admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:44.943357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a761c1-4c31-4ba3-b12f-dd8f648f446c admin admin] PATH_INFO: `/v3/projects/eacbef0dab9e453dae8f132d9edbab96/users/bc186f38b0e24b11bf97c9e83a04d89d/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:44.957653 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a761c1-4c31-4ba3-b12f-dd8f648f446c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eacbef0dab9e453dae8f132d9edbab96, user_id=bc186f38b0e24b11bf97c9e83a04d89d, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:44.960708 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a761c1-4c31-4ba3-b12f-dd8f648f446c admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:45.031026 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1449/2897] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:44 2022] PUT /identity/v3/projects/eacbef0dab9e453dae8f132d9edbab96/users/bc186f38b0e24b11bf97c9e83a04d89d/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:45.036081 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442647ea-e1ea-4791-b8b2-1e4c4dadb432 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:45.036519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442647ea-e1ea-4791-b8b2-1e4c4dadb432 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:45.036804 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442647ea-e1ea-4791-b8b2-1e4c4dadb432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:45.056602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-442647ea-e1ea-4791-b8b2-1e4c4dadb432 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:45.057807 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-442647ea-e1ea-4791-b8b2-1e4c4dadb432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:45.117663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1449/2898] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:45.127381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a761c1-4c31-4ba3-b12f-dd8f648f446c admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:45.129184 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-32a761c1-4c31-4ba3-b12f-dd8f648f446c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:45.186672 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a761c1-4c31-4ba3-b12f-dd8f648f446c admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:45.190664 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cb2be-5da7-40c5-b3d9-96fe14b70f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:45.191263 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cb2be-5da7-40c5-b3d9-96fe14b70f4d service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:45.191541 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cb2be-5da7-40c5-b3d9-96fe14b70f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:45.191811 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44cb2be-5da7-40c5-b3d9-96fe14b70f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:45.201588 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cb2be-5da7-40c5-b3d9-96fe14b70f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:45.203038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44cb2be-5da7-40c5-b3d9-96fe14b70f4d service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:45.213044 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1450/2899] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:46.621446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0e3b-4d04-4fe9-91a4-6f21710ce6ae None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:46.621996 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0e3b-4d04-4fe9-91a4-6f21710ce6ae None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:46.622100 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a0e3b-4d04-4fe9-91a4-6f21710ce6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:46.653904 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-782a0e3b-4d04-4fe9-91a4-6f21710ce6ae None None] MFA Rules not processed for user `bc186f38b0e24b11bf97c9e83a04d89d`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:46.655150 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-782a0e3b-4d04-4fe9-91a4-6f21710ce6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:46.688721 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:46.724849 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1450/2900] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:33:46.729920 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cb2be-5da7-40c5-b3d9-96fe14b70f4d service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:46.740078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44cb2be-5da7-40c5-b3d9-96fe14b70f4d service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:46.744896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3a83f-a0ac-4f39-a06b-0f395c6c93b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:46.745501 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3a83f-a0ac-4f39-a06b-0f395c6c93b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:46.745778 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3a83f-a0ac-4f39-a06b-0f395c6c93b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:46.746069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3a83f-a0ac-4f39-a06b-0f395c6c93b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:46.746676 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3a83f-a0ac-4f39-a06b-0f395c6c93b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:46.748527 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf3a83f-a0ac-4f39-a06b-0f395c6c93b6 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:46.849289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1451/2901] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:33:46 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:46.853474 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a0e3b-4d04-4fe9-91a4-6f21710ce6ae None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:46.867764 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a0e3b-4d04-4fe9-91a4-6f21710ce6ae None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:46.927922 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70dc14-af2f-488c-b9d1-24c2f48cb286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:46.928547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70dc14-af2f-488c-b9d1-24c2f48cb286 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:46.928828 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70dc14-af2f-488c-b9d1-24c2f48cb286 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:46.929102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70dc14-af2f-488c-b9d1-24c2f48cb286 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:46.929728 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70dc14-af2f-488c-b9d1-24c2f48cb286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:46.931674 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70dc14-af2f-488c-b9d1-24c2f48cb286 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:47.042662 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1451/2902] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:33:46 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:33:47.046562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3a83f-a0ac-4f39-a06b-0f395c6c93b6 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:47.057297 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3a83f-a0ac-4f39-a06b-0f395c6c93b6 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:47.061765 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef5028-715c-43a5-9666-69c8338a55c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:47.063004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef5028-715c-43a5-9666-69c8338a55c8 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:47.063004 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef5028-715c-43a5-9666-69c8338a55c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:47.063300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef5028-715c-43a5-9666-69c8338a55c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:47.063821 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef5028-715c-43a5-9666-69c8338a55c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:47.065386 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef5028-715c-43a5-9666-69c8338a55c8 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:47.076553 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1452/2903] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:47.080920 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70dc14-af2f-488c-b9d1-24c2f48cb286 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:47.091174 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70dc14-af2f-488c-b9d1-24c2f48cb286 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:47.095813 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf5117-e44b-40b6-9828-ed601263eb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:47.096455 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf5117-e44b-40b6-9828-ed601263eb02 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:47.096736 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf5117-e44b-40b6-9828-ed601263eb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:47.097009 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cf5117-e44b-40b6-9828-ed601263eb02 admin admin] PATH_INFO: `/v3/projects/82620a7f63e34350957b65c3d827eef1/users/ecd35a398eec45dfbcc3d85f03acd44d/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:47.112745 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf5117-e44b-40b6-9828-ed601263eb02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82620a7f63e34350957b65c3d827eef1, user_id=ecd35a398eec45dfbcc3d85f03acd44d, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:47.115441 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cf5117-e44b-40b6-9828-ed601263eb02 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:47.236582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1452/2904] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:47 2022] PUT /identity/v3/projects/82620a7f63e34350957b65c3d827eef1/users/ecd35a398eec45dfbcc3d85f03acd44d/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 09 17:33:47.240623 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef5028-715c-43a5-9666-69c8338a55c8 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:47.250774 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef5028-715c-43a5-9666-69c8338a55c8 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:47.306952 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc495d3f-77db-4064-bf9f-3b0a48ae1774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:47.307739 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc495d3f-77db-4064-bf9f-3b0a48ae1774 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:47.308029 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc495d3f-77db-4064-bf9f-3b0a48ae1774 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:47.308301 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc495d3f-77db-4064-bf9f-3b0a48ae1774 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:47.308939 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc495d3f-77db-4064-bf9f-3b0a48ae1774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:47.310544 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc495d3f-77db-4064-bf9f-3b0a48ae1774 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:47.315805 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:47.320776 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1453/2905] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:33:47.325115 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf5117-e44b-40b6-9828-ed601263eb02 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:47.335983 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cf5117-e44b-40b6-9828-ed601263eb02 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:47.340718 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9492d-a7a7-4c8a-a10b-eb5462de6c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:47.341349 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9492d-a7a7-4c8a-a10b-eb5462de6c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:47.341629 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9492d-a7a7-4c8a-a10b-eb5462de6c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:47.341898 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9492d-a7a7-4c8a-a10b-eb5462de6c96 admin admin] PATH_INFO: `/v3/projects/82620a7f63e34350957b65c3d827eef1/users/ecd35a398eec45dfbcc3d85f03acd44d/roles/68125a226d1d4e7c90d58b42cf2c1503` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:47.344176 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9492d-a7a7-4c8a-a10b-eb5462de6c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82620a7f63e34350957b65c3d827eef1, user_id=ecd35a398eec45dfbcc3d85f03acd44d, role_id=68125a226d1d4e7c90d58b42cf2c1503)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:47.346491 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d9492d-a7a7-4c8a-a10b-eb5462de6c96 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:47.357591 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1453/2906] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:33:47 2022] PUT /identity/v3/projects/82620a7f63e34350957b65c3d827eef1/users/ecd35a398eec45dfbcc3d85f03acd44d/roles/68125a226d1d4e7c90d58b42cf2c1503 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:33:48.667756 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055fd71-1b63-4391-8202-b09aa7e195d0 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:48.668387 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055fd71-1b63-4391-8202-b09aa7e195d0 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:48.668619 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f055fd71-1b63-4391-8202-b09aa7e195d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:48.699132 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-f055fd71-1b63-4391-8202-b09aa7e195d0 None None] MFA Rules not processed for user `ecd35a398eec45dfbcc3d85f03acd44d`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:33:48.700420 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f055fd71-1b63-4391-8202-b09aa7e195d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:48.770353 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1454/2907] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:33:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 09 17:33:48.782162 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9492d-a7a7-4c8a-a10b-eb5462de6c96 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:48.784075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-15d9492d-a7a7-4c8a-a10b-eb5462de6c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:48.841583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9492d-a7a7-4c8a-a10b-eb5462de6c96 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:48.845645 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7fa09-09c3-4f23-977a-4a0654c55b80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:48.846270 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7fa09-09c3-4f23-977a-4a0654c55b80 service cinder] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:48.846547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7fa09-09c3-4f23-977a-4a0654c55b80 service cinder] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:48.846844 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7fa09-09c3-4f23-977a-4a0654c55b80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:48.880446 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:33:48.903505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7fa09-09c3-4f23-977a-4a0654c55b80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:48.904926 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7fa09-09c3-4f23-977a-4a0654c55b80 service cinder] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:48.913634 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1454/2908] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:50.604561 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055fd71-1b63-4391-8202-b09aa7e195d0 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:33:50.606076 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-f055fd71-1b63-4391-8202-b09aa7e195d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:33:50.618540 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f055fd71-1b63-4391-8202-b09aa7e195d0 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:33:50.622127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde50c75-8224-4555-b46f-19285a422bb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:33:50.622731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde50c75-8224-4555-b46f-19285a422bb2 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:33:50.623023 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde50c75-8224-4555-b46f-19285a422bb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:33:50.623270 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde50c75-8224-4555-b46f-19285a422bb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:33:50.632160 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde50c75-8224-4555-b46f-19285a422bb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:33:50.633564 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde50c75-8224-4555-b46f-19285a422bb2 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:33:50.643416 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1455/2909] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:33:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:33:58.945299 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=11 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:00.638731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=40 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:06.783777 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7fa09-09c3-4f23-977a-4a0654c55b80 service cinder] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:06.790073 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-3dc7fa09-09c3-4f23-977a-4a0654c55b80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:06.869350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7fa09-09c3-4f23-977a-4a0654c55b80 service cinder] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:06.874627 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca024cf-8a02-4580-a129-33ee2d09ec5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:06.875227 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca024cf-8a02-4580-a129-33ee2d09ec5c service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:06.875500 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca024cf-8a02-4580-a129-33ee2d09ec5c service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:06.875769 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca024cf-8a02-4580-a129-33ee2d09ec5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:06.877572 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-9ca024cf-8a02-4580-a129-33ee2d09ec5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:06.891365 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca024cf-8a02-4580-a129-33ee2d09ec5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:06.895758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca024cf-8a02-4580-a129-33ee2d09ec5c service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:06.904552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1455/2910] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:07.452736 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde50c75-8224-4555-b46f-19285a422bb2 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:07.464105 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde50c75-8224-4555-b46f-19285a422bb2 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:07.469369 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad52b8e-1515-4795-aa99-6e60e3e0cfad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:07.469896 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad52b8e-1515-4795-aa99-6e60e3e0cfad service placement] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:07.470289 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad52b8e-1515-4795-aa99-6e60e3e0cfad service placement] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:07.470526 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad52b8e-1515-4795-aa99-6e60e3e0cfad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:07.472235 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-aad52b8e-1515-4795-aa99-6e60e3e0cfad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:07.484361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad52b8e-1515-4795-aa99-6e60e3e0cfad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:07.486121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad52b8e-1515-4795-aa99-6e60e3e0cfad service placement] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:07.500789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1456/2911] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:07.592450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca024cf-8a02-4580-a129-33ee2d09ec5c service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:07.656837 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca024cf-8a02-4580-a129-33ee2d09ec5c service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:07.661362 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:07.662002 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:07.662282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:07.662550 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] PATH_INFO: `/v3/users/bc186f38b0e24b11bf97c9e83a04d89d` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:07.663658 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc186f38b0e24b11bf97c9e83a04d89d)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:07.665547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:07.892799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc186f38b0e24b11bf97c9e83a04d89d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:07.963248 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc186f38b0e24b11bf97c9e83a04d89d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:07.971703 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc186f38b0e24b11bf97c9e83a04d89d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:07.984944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc186f38b0e24b11bf97c9e83a04d89d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:07.985653 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc186f38b0e24b11bf97c9e83a04d89d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:07.991855 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1456/2912] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:34:07 2022] DELETE /identity/v3/users/bc186f38b0e24b11bf97c9e83a04d89d => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:34:08.263667 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad52b8e-1515-4795-aa99-6e60e3e0cfad service placement] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:08.275097 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad52b8e-1515-4795-aa99-6e60e3e0cfad service placement] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:08.336313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:08.337016 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:08.337215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:08.337484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] PATH_INFO: `/v3/projects/eacbef0dab9e453dae8f132d9edbab96` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:08.338610 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=eacbef0dab9e453dae8f132d9edbab96)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:08.340978 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:08.462337 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eacbef0dab9e453dae8f132d9edbab96', 'request_id': 'req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:08.546313 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eacbef0dab9e453dae8f132d9edbab96', 'request_id': 'req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:08.553280 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eacbef0dab9e453dae8f132d9edbab96', 'request_id': 'req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:08.564077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] The token cache is being invalidate because project eacbef0dab9e453dae8f132d9edbab96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:34:08.564361 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:08.564996 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:08.566537 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1457/2913] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:34:08 2022] DELETE /identity/v3/projects/eacbef0dab9e453dae8f132d9edbab96 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:34:08.570695 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:08.572117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:08.585723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e7868-5153-465f-929b-c22bc579fb2e admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:08.590881 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:08.641753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:08.642442 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:08.642720 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:08.642985 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] PATH_INFO: `/v3/users/ecd35a398eec45dfbcc3d85f03acd44d` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:08.644136 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd35a398eec45dfbcc3d85f03acd44d)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:08.646066 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:08.888234 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd35a398eec45dfbcc3d85f03acd44d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:08.980682 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd35a398eec45dfbcc3d85f03acd44d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:08.988621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd35a398eec45dfbcc3d85f03acd44d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:09.001427 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd35a398eec45dfbcc3d85f03acd44d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:09.002195 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-ad575c30-e01f-4561-838b-89d12895f3e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd35a398eec45dfbcc3d85f03acd44d'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:09.008297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1457/2914] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:34:08 2022] DELETE /identity/v3/users/ecd35a398eec45dfbcc3d85f03acd44d => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:34:09.227851 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:09.234061 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:09.247252 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c1676-7d68-4bb0-9077-e95ff06ecf7a admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:09.307000 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:09.307773 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:09.308624 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:09.309077 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] PATH_INFO: `/v3/projects/82620a7f63e34350957b65c3d827eef1` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:09.310098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82620a7f63e34350957b65c3d827eef1)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:09.311969 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:09.333035 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:09.415199 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82620a7f63e34350957b65c3d827eef1', 'request_id': 'req-8c2758d1-335a-4a47-956b-21599e54d239'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:09.500426 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82620a7f63e34350957b65c3d827eef1', 'request_id': 'req-8c2758d1-335a-4a47-956b-21599e54d239'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:09.507849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82620a7f63e34350957b65c3d827eef1', 'request_id': 'req-8c2758d1-335a-4a47-956b-21599e54d239'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:09.518750 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] The token cache is being invalidate because project 82620a7f63e34350957b65c3d827eef1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:34:09.519038 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:09.519685 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-8c2758d1-335a-4a47-956b-21599e54d239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:09.521197 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1458/2915] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:34:09 2022] DELETE /identity/v3/projects/82620a7f63e34350957b65c3d827eef1 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:34:09.531656 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b861b-0578-4edc-940a-cd2260c5d040 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:09.531944 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b861b-0578-4edc-940a-cd2260c5d040 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:09.532220 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b861b-0578-4edc-940a-cd2260c5d040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:09.553621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-8e8b861b-0578-4edc-940a-cd2260c5d040 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:34:09.554865 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-8e8b861b-0578-4edc-940a-cd2260c5d040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:09.614805 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1458/2916] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:34:09.622793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a3aa0-97e6-49ab-9370-f3d9e97f7f62 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:09.623086 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a3aa0-97e6-49ab-9370-f3d9e97f7f62 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:09.623347 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a3aa0-97e6-49ab-9370-f3d9e97f7f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:09.642942 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-389a3aa0-97e6-49ab-9370-f3d9e97f7f62 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:34:09.644176 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-389a3aa0-97e6-49ab-9370-f3d9e97f7f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:09.652789 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1459/2917] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:34:09.657130 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b861b-0578-4edc-940a-cd2260c5d040 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:09.666747 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b861b-0578-4edc-940a-cd2260c5d040 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:09.675445 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a14df-369b-46c1-abe8-67a8313b6ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:09.676043 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a14df-369b-46c1-abe8-67a8313b6ffe admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:09.676313 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a14df-369b-46c1-abe8-67a8313b6ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:09.676573 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a14df-369b-46c1-abe8-67a8313b6ffe admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:09.677208 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a14df-369b-46c1-abe8-67a8313b6ffe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:34:09.677499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a14df-369b-46c1-abe8-67a8313b6ffe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:09.679108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a14df-369b-46c1-abe8-67a8313b6ffe admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:09.691450 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1459/2918] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 09 17:34:09.696190 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a3aa0-97e6-49ab-9370-f3d9e97f7f62 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:09.707484 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a3aa0-97e6-49ab-9370-f3d9e97f7f62 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:09.712196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597d0bb-bef8-4544-a00f-84ae1c5f1df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:09.712813 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597d0bb-bef8-4544-a00f-84ae1c5f1df3 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:09.713081 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597d0bb-bef8-4544-a00f-84ae1c5f1df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:09.713344 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5597d0bb-bef8-4544-a00f-84ae1c5f1df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:09.713986 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597d0bb-bef8-4544-a00f-84ae1c5f1df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:09.716341 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5597d0bb-bef8-4544-a00f-84ae1c5f1df3 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:09.810608 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1460/2919] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:34:09 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:34:09.815102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a14df-369b-46c1-abe8-67a8313b6ffe admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:09.825426 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a14df-369b-46c1-abe8-67a8313b6ffe admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:09.836758 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:09.880799 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5dcce-5056-4cb9-a5aa-fdb98c991dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:09.881391 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5dcce-5056-4cb9-a5aa-fdb98c991dda admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:09.881663 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5dcce-5056-4cb9-a5aa-fdb98c991dda admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:09.881930 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5dcce-5056-4cb9-a5aa-fdb98c991dda admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:09.882583 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5dcce-5056-4cb9-a5aa-fdb98c991dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:09.884468 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5dcce-5056-4cb9-a5aa-fdb98c991dda admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:09.963551 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1460/2920] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:34:09 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:34:09.968799 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597d0bb-bef8-4544-a00f-84ae1c5f1df3 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:09.979300 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5597d0bb-bef8-4544-a00f-84ae1c5f1df3 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:09.983878 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa91729-23e5-43a5-a4ad-5d0600b4e7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:09.984461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa91729-23e5-43a5-a4ad-5d0600b4e7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:09.984735 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa91729-23e5-43a5-a4ad-5d0600b4e7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:09.985039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa91729-23e5-43a5-a4ad-5d0600b4e7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:09.985665 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa91729-23e5-43a5-a4ad-5d0600b4e7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:09.987257 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa91729-23e5-43a5-a4ad-5d0600b4e7b0 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:09.997448 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1461/2921] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:34:10.003487 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5dcce-5056-4cb9-a5aa-fdb98c991dda admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:10.014465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5dcce-5056-4cb9-a5aa-fdb98c991dda admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:10.019179 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99a6fe-6747-4f67-a1d3-5dd492bf533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:10.019761 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99a6fe-6747-4f67-a1d3-5dd492bf533a admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:10.020033 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99a6fe-6747-4f67-a1d3-5dd492bf533a admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:10.020327 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99a6fe-6747-4f67-a1d3-5dd492bf533a admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:10.020951 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99a6fe-6747-4f67-a1d3-5dd492bf533a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:10.022850 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99a6fe-6747-4f67-a1d3-5dd492bf533a admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:10.034080 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1461/2922] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 09 17:34:10.038687 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa91729-23e5-43a5-a4ad-5d0600b4e7b0 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:10.048580 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa91729-23e5-43a5-a4ad-5d0600b4e7b0 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:10.053137 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49901b-4375-464d-be61-689f8d53e641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:10.053771 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49901b-4375-464d-be61-689f8d53e641 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:10.054078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49901b-4375-464d-be61-689f8d53e641 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:10.054351 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49901b-4375-464d-be61-689f8d53e641 admin admin] PATH_INFO: `/v3/projects/2349875bebaf41f4bf485339eb46d22f/users/5c56f8fd1eca43d3ac4009d9cdb1a516/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:10.069071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49901b-4375-464d-be61-689f8d53e641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2349875bebaf41f4bf485339eb46d22f, user_id=5c56f8fd1eca43d3ac4009d9cdb1a516, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:10.071650 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49901b-4375-464d-be61-689f8d53e641 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:10.187176 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1462/2923] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:34:10 2022] PUT /identity/v3/projects/2349875bebaf41f4bf485339eb46d22f/users/5c56f8fd1eca43d3ac4009d9cdb1a516/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:34:10.197793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768a690-0cf1-479d-bd1a-96cbfd5193ac None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:10.198102 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768a690-0cf1-479d-bd1a-96cbfd5193ac None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:10.198374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768a690-0cf1-479d-bd1a-96cbfd5193ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:10.227520 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-1768a690-0cf1-479d-bd1a-96cbfd5193ac None None] MFA Rules not processed for user `5c56f8fd1eca43d3ac4009d9cdb1a516`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:34:10.228723 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1768a690-0cf1-479d-bd1a-96cbfd5193ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:10.291905 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1462/2924] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:34:10.303127 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49901b-4375-464d-be61-689f8d53e641 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:10.305731 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-5f49901b-4375-464d-be61-689f8d53e641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:10.361838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49901b-4375-464d-be61-689f8d53e641 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:10.365508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000091e4-c9c3-49ba-a5b1-002e1a84d076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:10.366144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000091e4-c9c3-49ba-a5b1-002e1a84d076 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:10.366422 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000091e4-c9c3-49ba-a5b1-002e1a84d076 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:10.366689 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000091e4-c9c3-49ba-a5b1-002e1a84d076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:10.375678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000091e4-c9c3-49ba-a5b1-002e1a84d076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:10.377098 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000091e4-c9c3-49ba-a5b1-002e1a84d076 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:10.385994 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1463/2925] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:10.407661 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768a690-0cf1-479d-bd1a-96cbfd5193ac None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:10.468891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768a690-0cf1-479d-bd1a-96cbfd5193ac None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:10.473654 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:10.474291 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:10.474571 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:10.474841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] PATH_INFO: `/v3/users/5c56f8fd1eca43d3ac4009d9cdb1a516` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:10.475957 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c56f8fd1eca43d3ac4009d9cdb1a516)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:10.477848 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:10.552892 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:10.681458 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c56f8fd1eca43d3ac4009d9cdb1a516'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:10.763108 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c56f8fd1eca43d3ac4009d9cdb1a516'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:10.771160 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c56f8fd1eca43d3ac4009d9cdb1a516'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:10.782699 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c56f8fd1eca43d3ac4009d9cdb1a516'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:10.783374 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c56f8fd1eca43d3ac4009d9cdb1a516'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:10.789272 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1463/2926] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:34:10 2022] DELETE /identity/v3/users/5c56f8fd1eca43d3ac4009d9cdb1a516 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:34:10.794295 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3bd3b-ac25-45b8-b9fb-ac327200a19d None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:10.796706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3bd3b-ac25-45b8-b9fb-ac327200a19d None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:10.796706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3bd3b-ac25-45b8-b9fb-ac327200a19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:10.815285 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-53f3bd3b-ac25-45b8-b9fb-ac327200a19d None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:34:10.816250 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-53f3bd3b-ac25-45b8-b9fb-ac327200a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:10.859620 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:10.877793 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1464/2927] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:34:10.888754 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:10.890185 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:10.948127 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ac34a-431b-4ace-a7da-cf5d37ad7d2c admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:10.951690 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0e4c1-384b-4702-bbac-4775e9775ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:10.952278 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e4c1-384b-4702-bbac-4775e9775ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:10.952554 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e4c1-384b-4702-bbac-4775e9775ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:10.952822 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0e4c1-384b-4702-bbac-4775e9775ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:10.961836 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0e4c1-384b-4702-bbac-4775e9775ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:10.963282 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0e4c1-384b-4702-bbac-4775e9775ee1 service neutron] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:10.973235 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1464/2928] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:11.376791 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3bd3b-ac25-45b8-b9fb-ac327200a19d None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:11.378536 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-53f3bd3b-ac25-45b8-b9fb-ac327200a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:11.391513 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3bd3b-ac25-45b8-b9fb-ac327200a19d None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:11.395905 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:11.396531 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:11.396808 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:11.397078 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] PATH_INFO: `/v3/projects/2349875bebaf41f4bf485339eb46d22f` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:11.398268 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2349875bebaf41f4bf485339eb46d22f)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:11.400136 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:11.519498 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2349875bebaf41f4bf485339eb46d22f', 'request_id': 'req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:11.592294 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2349875bebaf41f4bf485339eb46d22f', 'request_id': 'req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:11.599181 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2349875bebaf41f4bf485339eb46d22f', 'request_id': 'req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:11.609558 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] The token cache is being invalidate because project 2349875bebaf41f4bf485339eb46d22f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:34:11.613622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:11.613622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-b2d5f33d-87dd-4393-958f-2bd71a67e0e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:34:11.613622 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1465/2929] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:34:11 2022] DELETE /identity/v3/projects/2349875bebaf41f4bf485339eb46d22f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:34:11.617025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a724b0-ba25-4c1c-9f27-e420a085089e None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:11.617498 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a724b0-ba25-4c1c-9f27-e420a085089e None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:11.617595 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a724b0-ba25-4c1c-9f27-e420a085089e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:11.638005 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-c9a724b0-ba25-4c1c-9f27-e420a085089e None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:34:11.639198 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-c9a724b0-ba25-4c1c-9f27-e420a085089e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:11.696254 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1465/2930] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:34:11.705534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7e29a-0317-499f-86c2-101d683591ce None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:11.705817 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7e29a-0317-499f-86c2-101d683591ce None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:11.706141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7e29a-0317-499f-86c2-101d683591ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:11.726112 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-6eb7e29a-0317-499f-86c2-101d683591ce None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:34:11.727340 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-6eb7e29a-0317-499f-86c2-101d683591ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:11.735707 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1466/2931] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:34:11.739808 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a724b0-ba25-4c1c-9f27-e420a085089e None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:11.749984 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a724b0-ba25-4c1c-9f27-e420a085089e None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:11.754552 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8727f65-c5c0-41ed-af38-f7bce5ce7630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:11.755141 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8727f65-c5c0-41ed-af38-f7bce5ce7630 admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:11.755416 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8727f65-c5c0-41ed-af38-f7bce5ce7630 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:11.755685 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8727f65-c5c0-41ed-af38-f7bce5ce7630 admin admin] PATH_INFO: `/v3/domains` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:11.756324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8727f65-c5c0-41ed-af38-f7bce5ce7630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86843) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 09 17:34:11.756621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8727f65-c5c0-41ed-af38-f7bce5ce7630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:11.758350 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8727f65-c5c0-41ed-af38-f7bce5ce7630 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:11.770413 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1466/2932] 10.222.0.79 () {58 vars in 1153 bytes} [Tue Aug 9 17:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 09 17:34:11.774937 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7e29a-0317-499f-86c2-101d683591ce None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:11.785381 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7e29a-0317-499f-86c2-101d683591ce None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:11.790198 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc120bc-d71e-4b3a-b00a-04c44fd69a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:11.790798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc120bc-d71e-4b3a-b00a-04c44fd69a61 admin admin] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:11.791079 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc120bc-d71e-4b3a-b00a-04c44fd69a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:11.791346 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc120bc-d71e-4b3a-b00a-04c44fd69a61 admin admin] PATH_INFO: `/v3/projects` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:11.791975 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc120bc-d71e-4b3a-b00a-04c44fd69a61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:11.794562 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc120bc-d71e-4b3a-b00a-04c44fd69a61 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:11.885064 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1467/2933] 10.222.0.79 () {60 vars in 1153 bytes} [Tue Aug 9 17:34:11 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:34:11.889311 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8727f65-c5c0-41ed-af38-f7bce5ce7630 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:11.899536 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8727f65-c5c0-41ed-af38-f7bce5ce7630 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:11.931177 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:11.958870 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b9b9d-e4cd-4976-a1cb-6d83097fa872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:11.959465 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b9b9d-e4cd-4976-a1cb-6d83097fa872 admin admin] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:11.959739 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b9b9d-e4cd-4976-a1cb-6d83097fa872 admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:11.960003 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b9b9d-e4cd-4976-a1cb-6d83097fa872 admin admin] PATH_INFO: `/v3/users` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:11.960626 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b9b9d-e4cd-4976-a1cb-6d83097fa872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:11.962547 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b9b9d-e4cd-4976-a1cb-6d83097fa872 admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:12.055334 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1467/2934] 10.222.0.79 () {60 vars in 1144 bytes} [Tue Aug 9 17:34:11 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 09 17:34:12.059718 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc120bc-d71e-4b3a-b00a-04c44fd69a61 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:12.070678 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc120bc-d71e-4b3a-b00a-04c44fd69a61 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:12.075284 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc362e-d806-4056-be06-31d17a137b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:12.075869 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc362e-d806-4056-be06-31d17a137b14 admin admin] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:12.076144 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc362e-d806-4056-be06-31d17a137b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:12.076406 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc362e-d806-4056-be06-31d17a137b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:12.077040 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc362e-d806-4056-be06-31d17a137b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:12.078643 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc362e-d806-4056-be06-31d17a137b14 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:12.088355 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1468/2935] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:34:12.093015 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b9b9d-e4cd-4976-a1cb-6d83097fa872 admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:12.106324 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b9b9d-e4cd-4976-a1cb-6d83097fa872 admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:12.111475 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64d482-b3c9-4416-ba81-ecae3b58f48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:12.112088 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d482-b3c9-4416-ba81-ecae3b58f48d admin admin] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:12.112344 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d482-b3c9-4416-ba81-ecae3b58f48d admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:12.112621 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d482-b3c9-4416-ba81-ecae3b58f48d admin admin] PATH_INFO: `/v3/roles` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:12.113312 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64d482-b3c9-4416-ba81-ecae3b58f48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:12.114927 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64d482-b3c9-4416-ba81-ecae3b58f48d admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:12.124841 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1468/2936] 10.222.0.79 () {58 vars in 1122 bytes} [Tue Aug 9 17:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 09 17:34:12.129110 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc362e-d806-4056-be06-31d17a137b14 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:12.139636 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc362e-d806-4056-be06-31d17a137b14 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:12.144292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21644cc5-73db-46e6-b425-d5c1b989d819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:12.144935 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21644cc5-73db-46e6-b425-d5c1b989d819 admin admin] REQUEST_METHOD: `PUT` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:12.145215 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21644cc5-73db-46e6-b425-d5c1b989d819 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:12.145486 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21644cc5-73db-46e6-b425-d5c1b989d819 admin admin] PATH_INFO: `/v3/projects/e8df5e5606624ae2858c6f7072f96696/users/4f05da5ca5114446a1aef8f119f17a83/roles/2490390ea61f4cf594df3587e7674650` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:12.161559 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21644cc5-73db-46e6-b425-d5c1b989d819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8df5e5606624ae2858c6f7072f96696, user_id=4f05da5ca5114446a1aef8f119f17a83, role_id=2490390ea61f4cf594df3587e7674650)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:12.164196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21644cc5-73db-46e6-b425-d5c1b989d819 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:12.255020 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1469/2937] 10.222.0.79 () {60 vars in 1483 bytes} [Tue Aug 9 17:34:12 2022] PUT /identity/v3/projects/e8df5e5606624ae2858c6f7072f96696/users/4f05da5ca5114446a1aef8f119f17a83/roles/2490390ea61f4cf594df3587e7674650 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:34:12.260297 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819add1-b76f-4d7f-8d6a-c9e26913c375 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:12.260618 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819add1-b76f-4d7f-8d6a-c9e26913c375 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:12.260891 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819add1-b76f-4d7f-8d6a-c9e26913c375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:12.280694 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-2819add1-b76f-4d7f-8d6a-c9e26913c375 None None] MFA Rules not processed for user `ea8ba450a8204c87830e1b4dce2332ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:34:12.281897 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-2819add1-b76f-4d7f-8d6a-c9e26913c375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:12.338094 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1469/2938] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:34:12.348123 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21644cc5-73db-46e6-b425-d5c1b989d819 admin admin] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:12.349821 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-21644cc5-73db-46e6-b425-d5c1b989d819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:12.409590 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21644cc5-73db-46e6-b425-d5c1b989d819 admin admin] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:12.413191 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01551b2e-cd44-4e53-87fe-420eb950e1c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:12.413868 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01551b2e-cd44-4e53-87fe-420eb950e1c9 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:12.414168 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01551b2e-cd44-4e53-87fe-420eb950e1c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:12.414433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01551b2e-cd44-4e53-87fe-420eb950e1c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:12.423433 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01551b2e-cd44-4e53-87fe-420eb950e1c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:12.424838 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01551b2e-cd44-4e53-87fe-420eb950e1c9 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:12.434277 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1470/2939] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:13.490343 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda98b6-48e9-4ec7-b6a2-57c473d48f7b None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:13.490929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda98b6-48e9-4ec7-b6a2-57c473d48f7b None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:13.490929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda98b6-48e9-4ec7-b6a2-57c473d48f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:13.520910 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-1eda98b6-48e9-4ec7-b6a2-57c473d48f7b None None] MFA Rules not processed for user `4f05da5ca5114446a1aef8f119f17a83`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:34:13.522187 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-1eda98b6-48e9-4ec7-b6a2-57c473d48f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:13.595582 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1470/2940] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:34:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:34:13.606949 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01551b2e-cd44-4e53-87fe-420eb950e1c9 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:13.608401 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-01551b2e-cd44-4e53-87fe-420eb950e1c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:13.620469 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:13.670096 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01551b2e-cd44-4e53-87fe-420eb950e1c9 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:13.674453 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf8fc8-9e03-4be3-941d-4961e03bc478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '289e7caceab944e9b90f2e9298fb2cf1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:13.675071 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf8fc8-9e03-4be3-941d-4961e03bc478 service cinder] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:13.675333 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf8fc8-9e03-4be3-941d-4961e03bc478 service cinder] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:13.675599 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf8fc8-9e03-4be3-941d-4961e03bc478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:13.684069 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf8fc8-9e03-4be3-941d-4961e03bc478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:13.685490 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf8fc8-9e03-4be3-941d-4961e03bc478 service cinder] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:13.695121 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1471/2941] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:15.416481 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa019185-37c4-4438-9aa2-0665d919d7d4 None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:15.417038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa019185-37c4-4438-9aa2-0665d919d7d4 None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:15.417038 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa019185-37c4-4438-9aa2-0665d919d7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:15.438258 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-aa019185-37c4-4438-9aa2-0665d919d7d4 None None] MFA Rules not processed for user `4f05da5ca5114446a1aef8f119f17a83`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:34:15.439774 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-aa019185-37c4-4438-9aa2-0665d919d7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:15.446602 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1471/2942] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:34:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:34:15.685508 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf8fc8-9e03-4be3-941d-4961e03bc478 service cinder] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:15.687053 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-0acf8fc8-9e03-4be3-941d-4961e03bc478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:15.700019 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf8fc8-9e03-4be3-941d-4961e03bc478 service cinder] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:15.703647 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc926aa8-d380-49e2-948c-99e14cc110d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:15.704229 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc926aa8-d380-49e2-948c-99e14cc110d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:15.704503 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc926aa8-d380-49e2-948c-99e14cc110d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:15.704777 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc926aa8-d380-49e2-948c-99e14cc110d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:15.706491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-bc926aa8-d380-49e2-948c-99e14cc110d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:15.717884 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc926aa8-d380-49e2-948c-99e14cc110d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:15.719320 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc926aa8-d380-49e2-948c-99e14cc110d9 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:15.727471 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1472/2943] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:16.437029 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa019185-37c4-4438-9aa2-0665d919d7d4 None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:16.439027 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-aa019185-37c4-4438-9aa2-0665d919d7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:16.481060 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:16.510619 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa019185-37c4-4438-9aa2-0665d919d7d4 None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:16.516471 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74f82b-ba77-488c-a31c-75d73793c4eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:16.517075 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74f82b-ba77-488c-a31c-75d73793c4eb service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:16.517354 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74f82b-ba77-488c-a31c-75d73793c4eb service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:16.517648 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74f82b-ba77-488c-a31c-75d73793c4eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:16.519482 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4e74f82b-ba77-488c-a31c-75d73793c4eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:16.533047 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74f82b-ba77-488c-a31c-75d73793c4eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:16.534667 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74f82b-ba77-488c-a31c-75d73793c4eb service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:16.544681 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1472/2944] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:19.548156 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc926aa8-d380-49e2-948c-99e14cc110d9 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:19.557020 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc926aa8-d380-49e2-948c-99e14cc110d9 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:19.566292 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30873e-9f6c-43f5-9a63-7d3a91f14f30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2f11dcf2c3c41a0b435a90af74c2e5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:19.566898 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30873e-9f6c-43f5-9a63-7d3a91f14f30 service neutron] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:19.567217 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30873e-9f6c-43f5-9a63-7d3a91f14f30 service neutron] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:19.567442 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30873e-9f6c-43f5-9a63-7d3a91f14f30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:19.569304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-da30873e-9f6c-43f5-9a63-7d3a91f14f30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:19.580358 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30873e-9f6c-43f5-9a63-7d3a91f14f30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:19.581798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30873e-9f6c-43f5-9a63-7d3a91f14f30 service neutron] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:19.590124 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1473/2945] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:20.371793 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74f82b-ba77-488c-a31c-75d73793c4eb service placement] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:20.373800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-4e74f82b-ba77-488c-a31c-75d73793c4eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:20.457765 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74f82b-ba77-488c-a31c-75d73793c4eb service placement] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:20.463415 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edbeb17-e42a-4f33-90c3-9c2da3c3238e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:20.464037 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbeb17-e42a-4f33-90c3-9c2da3c3238e service nova] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:20.464316 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbeb17-e42a-4f33-90c3-9c2da3c3238e service nova] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:20.464684 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edbeb17-e42a-4f33-90c3-9c2da3c3238e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:20.466598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-5edbeb17-e42a-4f33-90c3-9c2da3c3238e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:20.484505 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbeb17-e42a-4f33-90c3-9c2da3c3238e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:20.493152 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edbeb17-e42a-4f33-90c3-9c2da3c3238e service nova] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:20.498527 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1473/2946] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:26.769634 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30873e-9f6c-43f5-9a63-7d3a91f14f30 service neutron] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:34:26.780965 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30873e-9f6c-43f5-9a63-7d3a91f14f30 service neutron] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:34:26.786506 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b0c3a-9a78-4f18-b201-c20ae7a9a8b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d17aa4b23e04290b6c8b1924b0dcf32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:34:26.787088 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b0c3a-9a78-4f18-b201-c20ae7a9a8b0 service nova] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:26.787357 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b0c3a-9a78-4f18-b201-c20ae7a9a8b0 service nova] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:26.787621 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b0c3a-9a78-4f18-b201-c20ae7a9a8b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:26.789304 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-919b0c3a-9a78-4f18-b201-c20ae7a9a8b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:26.803309 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b0c3a-9a78-4f18-b201-c20ae7a9a8b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:34:26.804847 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b0c3a-9a78-4f18-b201-c20ae7a9a8b0 service nova] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:34:26.815670 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1474/2947] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:34:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 09 17:34:30.492770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=50 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:35.303054 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab3475-588d-4ce6-be07-e32a4b1bc7ad None None] REQUEST_METHOD: `POST` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:34:35.305472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab3475-588d-4ce6-be07-e32a4b1bc7ad None None] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:34:35.305472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab3475-588d-4ce6-be07-e32a4b1bc7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:34:35.323275 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.auth.core [None req-cdab3475-588d-4ce6-be07-e32a4b1bc7ad None None] MFA Rules not processed for user `4f05da5ca5114446a1aef8f119f17a83`. Rule list: `[]` (Enabled: `True`). {{(pid=86843) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:34:35.324460 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cdab3475-588d-4ce6-be07-e32a4b1bc7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:34:35.331929 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1474/2948] 10.222.0.79 () {58 vars in 958 bytes} [Tue Aug 9 17:34:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:34:36.811070 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=40 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:34:45.317918 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=3 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:35:02.195865 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d144a0-9280-40f6-84b0-c84dfd59840b None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:35:02.196374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d144a0-9280-40f6-84b0-c84dfd59840b None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:35:02.196374 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d144a0-9280-40f6-84b0-c84dfd59840b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:35:02.216530 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-c5d144a0-9280-40f6-84b0-c84dfd59840b None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:35:02.218281 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-c5d144a0-9280-40f6-84b0-c84dfd59840b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:35:02.274706 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1475/2949] 10.222.0.79 () {58 vars in 1009 bytes} [Tue Aug 9 17:35:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:35:02.434071 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab3475-588d-4ce6-be07-e32a4b1bc7ad None None] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:35:02.435499 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-cdab3475-588d-4ce6-be07-e32a4b1bc7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:35:02.507529 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab3475-588d-4ce6-be07-e32a4b1bc7ad None None] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:35:02.513088 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126796a-9907-4f09-9eae-cb44db01c1b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:35:02.513689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126796a-9907-4f09-9eae-cb44db01c1b8 service swift] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:35:02.514017 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126796a-9907-4f09-9eae-cb44db01c1b8 service swift] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:35:02.514290 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126796a-9907-4f09-9eae-cb44db01c1b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:35:02.522757 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126796a-9907-4f09-9eae-cb44db01c1b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:35:02.524322 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126796a-9907-4f09-9eae-cb44db01c1b8 service swift] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:35:02.533025 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1475/2950] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 17:35:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:35:12.267989 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=25 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:35:12.529683 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=32 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:35:19.609491 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d695c8-4d6f-4f52-9d14-e92bf8da2444 None None] REQUEST_METHOD: `POST` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:35:19.610055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d695c8-4d6f-4f52-9d14-e92bf8da2444 None None] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:35:19.610055 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d695c8-4d6f-4f52-9d14-e92bf8da2444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:35:19.644837 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.auth.core [None req-b6d695c8-4d6f-4f52-9d14-e92bf8da2444 None None] MFA Rules not processed for user `a010ea3529644fca951a7eef28dbe97e`. Rule list: `[]` (Enabled: `True`). {{(pid=86844) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 09 17:35:19.647264 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b6d695c8-4d6f-4f52-9d14-e92bf8da2444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:35:19.658141 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1476/2951] 10.222.0.79 () {58 vars in 967 bytes} [Tue Aug 9 17:35:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 09 17:35:19.670753 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126796a-9907-4f09-9eae-cb44db01c1b8 service swift] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:35:19.681782 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126796a-9907-4f09-9eae-cb44db01c1b8 service swift] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:35:19.686895 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a94a6f-aff2-4d4b-b25a-33545313b93f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35371b488c474b7f851af16bdad34f69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:35:19.687513 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a94a6f-aff2-4d4b-b25a-33545313b93f service swift] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:35:19.687795 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a94a6f-aff2-4d4b-b25a-33545313b93f service swift] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:35:19.688064 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a94a6f-aff2-4d4b-b25a-33545313b93f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:35:19.697019 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a94a6f-aff2-4d4b-b25a-33545313b93f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:35:19.698670 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a94a6f-aff2-4d4b-b25a-33545313b93f service swift] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:35:19.707406 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1476/2952] 10.222.0.79 () {60 vars in 1384 bytes} [Tue Aug 9 17:35:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 09 17:35:22.739041 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d695c8-4d6f-4f52-9d14-e92bf8da2444 None None] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:35:22.741103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-b6d695c8-4d6f-4f52-9d14-e92bf8da2444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:35:22.756970 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d695c8-4d6f-4f52-9d14-e92bf8da2444 None None] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:35:22.762534 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507697bc-0e32-4bbf-acc5-78fa156593ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:35:22.763261 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507697bc-0e32-4bbf-acc5-78fa156593ab service placement] REQUEST_METHOD: `GET` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:35:22.763557 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507697bc-0e32-4bbf-acc5-78fa156593ab service placement] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:35:22.763848 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507697bc-0e32-4bbf-acc5-78fa156593ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:35:22.765732 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.fernet_utils [None req-507697bc-0e32-4bbf-acc5-78fa156593ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86844) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:35:22.777787 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507697bc-0e32-4bbf-acc5-78fa156593ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:35:22.779402 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507697bc-0e32-4bbf-acc5-78fa156593ab service placement] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:35:22.790461 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1477/2953] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:35:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:35:23.056192 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a94a6f-aff2-4d4b-b25a-33545313b93f service swift] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:35:23.067472 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a94a6f-aff2-4d4b-b25a-33545313b93f service swift] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:35:23.077800 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:35:23.078517 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] REQUEST_METHOD: `DELETE` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:35:23.078787 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:35:23.079055 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] PATH_INFO: `/v3/users/4f05da5ca5114446a1aef8f119f17a83` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:35:23.080519 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f05da5ca5114446a1aef8f119f17a83)` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:35:23.082452 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:35:23.234535 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f05da5ca5114446a1aef8f119f17a83'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:35:23.293105 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f05da5ca5114446a1aef8f119f17a83'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:35:23.300673 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f05da5ca5114446a1aef8f119f17a83'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:35:23.312743 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f05da5ca5114446a1aef8f119f17a83'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:35:23.313471 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.notifications [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f05da5ca5114446a1aef8f119f17a83'} {{(pid=86843) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:35:23.319561 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1477/2954] 10.222.0.79 () {58 vars in 1224 bytes} [Tue Aug 9 17:35:23 2022] DELETE /identity/v3/users/4f05da5ca5114446a1aef8f119f17a83 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:35:23.557798 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507697bc-0e32-4bbf-acc5-78fa156593ab service placement] Authenticating user token {{(pid=86844) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:35:23.573196 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507697bc-0e32-4bbf-acc5-78fa156593ab service placement] Validating token access rules against request {{(pid=86844) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:35:23.635482 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea8ba450a8204c87830e1b4dce2332ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e0a53b8ab06c430ea7305247e9df6f92', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86844) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:35:23.636113 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:35:23.636385 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:35:23.636656 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] PATH_INFO: `/v3/projects/e8df5e5606624ae2858c6f7072f96696` {{(pid=86844) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:35:23.637781 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8df5e5606624ae2858c6f7072f96696)` {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:35:23.639709 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] RBAC: Authorization granted {{(pid=86844) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:35:23.786192 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8df5e5606624ae2858c6f7072f96696', 'request_id': 'req-cf0d641c-0443-4152-98c8-7e1f3271e0b3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:35:23.864727 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8df5e5606624ae2858c6f7072f96696', 'request_id': 'req-cf0d641c-0443-4152-98c8-7e1f3271e0b3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:35:23.874103 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8df5e5606624ae2858c6f7072f96696', 'request_id': 'req-cf0d641c-0443-4152-98c8-7e1f3271e0b3'} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:35:23.886563 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] The token cache is being invalidate because project e8df5e5606624ae2858c6f7072f96696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86844) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 09 17:35:23.886849 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:35:23.887514 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG keystone.notifications [None req-cf0d641c-0443-4152-98c8-7e1f3271e0b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86844) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 09 17:35:23.889039 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: [pid: 86844|app: 0|req: 1478/2955] 10.222.0.79 () {58 vars in 1233 bytes} [Tue Aug 9 17:35:23 2022] DELETE /identity/v3/projects/e8df5e5606624ae2858c6f7072f96696 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 09 17:35:33.315902 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:35:33.882803 dv-cinder-831247-22-iscsi devstack@keystone.service[86844]: DEBUG dbcounter [-] [86844] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86844) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 09 17:40:28.827688 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] Authenticating user token {{(pid=86843) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 09 17:40:28.829099 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:40:28.946096 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f9773-b43d-4bc5-bd01-9b67dd4027db admin admin] Validating token access rules against request {{(pid=86843) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 09 17:40:29.015256 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c1645-4dd5-4dc3-becc-625bacaeeb2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7bc938254514471801c88761525e883', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ec40a57f27e42ae81c45450e2297664', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86843) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 09 17:40:29.015770 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c1645-4dd5-4dc3-becc-625bacaeeb2f service placement] REQUEST_METHOD: `GET` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 09 17:40:29.016042 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c1645-4dd5-4dc3-becc-625bacaeeb2f service placement] SCRIPT_NAME: `/identity` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 09 17:40:29.016314 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c1645-4dd5-4dc3-becc-625bacaeeb2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86843) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 09 17:40:29.018117 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.fernet_utils [None req-829c1645-4dd5-4dc3-becc-625bacaeeb2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86843) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 09 17:40:29.030310 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c1645-4dd5-4dc3-becc-625bacaeeb2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 09 17:40:29.032044 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c1645-4dd5-4dc3-becc-625bacaeeb2f service placement] RBAC: Authorization granted {{(pid=86843) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 09 17:40:29.043689 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: [pid: 86843|app: 0|req: 1478/2956] 10.222.0.79 () {60 vars in 1365 bytes} [Tue Aug 9 17:40:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 09 17:40:39.038598 dv-cinder-831247-22-iscsi devstack@keystone.service[86843]: DEBUG dbcounter [-] [86843] Writing DB stats keystone:SELECT=74 {{(pid=86843) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}